Yeda AI Tips · #204

Open Weights Isn't Local, and Isn't Free

The thumbnails all promise the same thing: point the best coding harness on the planet at an open model, run it free, local, and open source, and stop paying the frontier labs forever. Most of that is real. But three different ideas get quietly welded into one, and untangling them is the whole tip. Open weights is not the same as local, and neither one means free. Get the three straight and you'll know exactly what an open model buys you before you rip out your setup.

Open weights: the model file is yours

Open weights means the trained model — the actual file of parameters — is yours to keep. You can download it, inspect how it was tuned, fine-tune it on your own data, and self-host it. Nobody can reach into your servers and switch it off. On a hub like Hugging Face, that file lives in a public model repository anyone can pull down with a client library, which is what "open" concretely looks like: the checkpoint is hosted, versioned, and shareable, not locked behind a vendor's API.

That is genuinely different from a closed model, where you only ever rent access through an endpoint and never see the weights. But "the file is yours" says nothing about whether your laptop can run it.

Local: the hardware laughs at you

Here is where the fantasy meets the hardware. One open frontier model ships at roughly 2.8 trillion parameters. Even at 4-bit quantization — packing each parameter into about half a byte — that's around 1.4 TB just for the raw weights, before any context or runtime overhead.

Now price the hardware. The largest Mac Studio tops out near 512 GB. Four H100 GPUs give you about 320 GB. Even an 8-GPU server at roughly 1.44 TB barely clears the floor, with nothing left over for the context window. Running the full model on your own machine isn't a desktop project — it's a data-center project. So when a thumbnail says "local," what it almost always means is running the open model on a provider's GPUs, on someone else's hardware, not yours.

One reason a model that huge can run at all is its mixture-of-experts design: it's enormous on disk, but only a fraction of its experts activate for any given token. It's lean when it actually runs and gigantic when it sits at rest — which is exactly why it fits in a rented server but never on a desk.

Free: usually a launch promo

"Free" is the third slippery word. At launch, some providers and platforms hand out promotional credits and trial tiers, so for light use you genuinely can start driving an open model at zero cost. That's real — but it's free to start, not free forever. A hosted open model still charges per token: on one such model, roughly $3 per million input tokens and about $15 per million output. Cheaper than the frontier, often by a lot, but not zero. And because these models tend to be token-hungry and slower, the sticker price per million can flatter the real bill — measure tokens actually spent, not the rate card.

So what does open weights actually buy you?

Control and durability. If you have the hardware or a rented server, you can self-host for privacy or compliance — your data never leaves your walls. You're not locked to one vendor's roadmap or pricing. And crucially, an open model can't be pulled out from under you. Remember that a closed frontier model was taken offline for days over an export-control scare; you can't switch off a file already sitting on ten thousand hard drives. That durability is the real meaning of "open" here, even when the model never touches your desk.

Takeaway

Open weights, local, and free are three separate claims. Open weights is almost always true — the file is genuinely yours to keep, inspect, and self-host. Local is usually marketing for "on a provider's GPUs," because a 1.4 TB model won't fit on a laptop. Free is usually a launch promo, not forever. Buy the open model for control and durability, and treat "local" as "someone else's data center" unless you actually own the rack.

Resources

Building an AI feature? Yeda AI designs, audits, and ships production LLM systems.

Talk to us · Read the blog