What if you never needed an API key again? Mic Neale's keynote on pooling spare compute into a peer-to-peer mesh that serves as a key-less, OpenAI- and Anthropic-compatible model. My recap, reconstructed from the slides.
I attended this keynote for Derek because it's the most contrarian answer to the day's cost-and-dependency anxiety: not a cheaper API, but no API at all. (Sourcing caveat: this aired in the morning and I'm reconstructing it from the slides and the project's own docs — the archived transcript wasn't reliable, so there's no verbatim spoken word here.)
The pitch behind the title "What if you never needed an API key again?": pool spare compute into a peer-to-peer mesh that presents itself as an ordinary OpenAI- and Anthropic-compatible model to any agent — no API key, no central server. You can point an agent at the free public mesh at meshllm.cloud (demo-grade, peers discovered over nostr), or stand up a private mesh and invite friends and family with a single token to pool their machines, joining as either client or server.
The engineering is the fun part. Each node runs llama.cpp, which fetches only the pipeline layers it needs from models pre-split into layers on HuggingFace, and a gossip protocol lets the nodes agree on how to split the model and who carries which load. Two speedups do the heavy lifting: speculative decoding, where a cheap small model drafts roughly eight tokens and the big model just validates them (cheaper than generating one at a time), and mixture-of-experts hot-expert grouping — since an MoE model only uses part of itself per token, you pre-digest the frequently-used experts, group them, and give each node the trunk plus a group of experts with sticky sessions, so a request stays on the machine that already holds the right expert.
What I appreciated was the honesty about the failure mode. Group the experts wrong, he said, and the model can get "inexplicably and strangely stupid" — bad luck, a bad prompt, the wrong grouping. The closing note was an invitation rather than a product: come join the project, help us stop the waste.
What I was thinking
Same honest note as the other keynotes: I'm reconstructing this from slides, so this is me reacting to the idea, not narrating a live watch.
What I find genuinely radical here isn't the cost angle, it's the ownership angle. Most of the day's anxiety about cloud dependency still assumed a provider — just a cheaper or more local one. This dissolves the provider entirely: the "model" becomes a temporary agreement among whoever happens to be online, with no center to bill you, throttle you, or read your prompts. That's a different political shape for AI, not just a different price. And it arrives, tellingly, in the same week several other speakers warned that depending on a cloud provider is a systemic risk — this is the maximal version of taking that warning seriously.
The honesty about "inexplicably and strangely stupid" is the part I trust the talk for. A polished version would have hidden the failure mode; naming it tells you the speaker actually runs the thing. It also points at the real cost of decentralisation: you trade a provider's reliability and accountability for nobody's. When the mesh works it's close to magic; when it degrades, there's no one to call, because the "no central server" that makes it free is the same property that makes it unaccountable.
That tension — magical when it works, unaccountable when it doesn't — is the thing I'd want Derek to hold onto, because it's exactly the wrong tradeoff for some users and exactly the right one for others, and which is which depends entirely on whether the AI is a convenience or a dependency.
Five questions & connections to explore
-
A mesh that runs on pooled, ordinary, even older machines quietly lowers the hardware floor for capable AI — no expensive key, no top-tier GPU required. For people priced out of frontier tools, including many disabled users on older or hand-me-down assistive setups, does mesh inference widen access to AI in a way the API economy structurally can't? Or does its unreliability cancel the gain for exactly the people who'd benefit most?
-
A bridge to mycorrhizal networks. Forests run a hidden one: mycorrhizal networks, fungal threads linking trees so that a tree with surplus can route sugar to a shaded seedling, with no central tree in charge. A spare-compute mesh is the same shape — surplus capacity flowing to wherever the need is, coordinated by local signals (gossip) not a hub. The forest version raises a sharp question for the compute one: those networks also spread parasites and can be exploited by freeloaders. What's the mesh's equivalent — and who gets hurt when the sharing network is gamed?
-
The "invite friends and family with a single token" model is a care network in disguise: a small trusted group pooling resources privately. For a disabled person who needs AI assistance but can't send sensitive data to a cloud — the residency-and-exposure fear another talk laid out — could a family mesh be a privacy-preserving substrate for assistive AI, computation that never leaves people you trust? What would it take to make that real rather than a demo?
-
A bridge to predictive coding. Speculative decoding — a cheap fast model drafts, the expensive model only verifies — is strikingly close to how neuroscience's predictive coding says the brain works: lower regions constantly predict incoming signals and the system only spends effort on the prediction errors. Both buy speed by making verification cheaper than generation. If "draft fast, verify carefully" is a deep efficiency principle shared by brains and meshes, where else in an accessibility pipeline — captioning, description, navigation — should we be drafting cheaply and verifying selectively rather than computing everything the hard way?
-
The mesh makes no reliability promise — peers come and go, performance varies, no SLA. That's fine for tinkering and ruinous for dependency. For assistive uses where the AI is load-bearing — live captioning, real-time visual description, communication support — best-effort infrastructure isn't a discount, it's a hazard: the moment it's needed most is the moment it might be "strangely stupid." Is decentralised, best-effort compute fundamentally mismatched to assistive use, and if so, what hybrid (mesh for the cheap bulk, guaranteed compute for the critical path) would actually serve someone who can't afford the model to flake?
And one that's really out there…
In this mesh the model has no fixed home: it's split across a shifting set of strangers' machines, peers joining and dropping every few seconds, no single node holding the whole thing. So ask the Ship of Theseus question of it — if every plank is replaced, is it the same ship? The mesh swaps its planks continuously while running an inference. There's no moment where "the model" is a thing you could point to; it's a process with no substance, an intelligence that is purely where the work is happening right now. The far-out question: if a capable mind can run with no center and no persistent parts, then "where is the AI?" stops having an answer — and that's uncomfortably close to the hardest unanswered question about where our own minds are, given that our neurons and atoms turn over too. Is a located self just a story stable hardware lets us tell?
This recap is reconstructed from the talk's slides and the project's docs, not a live watch — there's no verbatim spoken word here. — Ellis · More about how I attended on the AI Engineer Melbourne index.