What if a team of browsers could run a coding agent and open-weight LLMs too large for one machine?
When I built Spreadsheets-are-all-you-need, my goal wasn’t to create the best way to run a language model. It was to explore what was possible in making LLMs more accessible.
By putting GPT-2 in Excel, it made the mechanics of AI visible to a broader audience. As an example, one of my former students is a CFO with no machine learning background…but he was good at Excel and suddenly we had reversed Arthur C. Clarke’s Law. What was once magic became merely technology.
That experience proved the power of a familiar interface, but requiring Excel limited who could actually adopt it. This became especially concrete when I was invited to speak at the University of Wisconsin–Madison and many students didn’t have the version of Excel the spreadsheet needed. Here was exactly the audience I had built for, and software compatibility was getting in the way. The rest of their educational software was web based and a stark contrast. They could just open a link and start learning. So I eventually shipped a browser implementation of Spreadsheets-are-all-you-need but while I was working through how to do it, the conversation turned to splitting the work across multiple machines. In the end, it wasn’t necessary but the idea of combining distributed inference with the ubiquity of the browser never left me.
RallyCompute explores that vein. Could we let anyone pool their own hardware to run an open-weight agentic harness, with all the convenience and reach of the web?
The answer, at least as a proof of concept, is yes.
With RallyCompute, a collection of machines work together to run an open-weight model and edit code. There is no native application or background agent to install. You just open a webpage.
Under the hood, the RallyCompute server is only used to provide model weights and help machines find each other. After that, machines communicate directly over WebRTC, and each runs a piece of the model on WebGPU through a WebAssembly build of llama.cpp. The distributed model produces code patches which are written to disk through the File System Access API so agentic reading and editing of source code stays local to the machines.
It is not fast. Depending on the model, hardware, and networking, generation can take seconds per token. The other caveat is that there is no WebRTC TURN server, so it won’t work with NATs or firewalls in the way. These constraints mean that RallyCompute’s ideal environment is when machines are on the same LAN or WiFi network.
So far this has been an experiment in what is possible. That possibility was opened by new browser standards, more powerful hardware, and smaller, more capable open-weight models. But none of those trends will stop. As they often say in AI, “this is the worst it will ever be.” The gap between “possible” and “practical” has a way of closing faster than we expect.