Yeda AI Tips · #206

Point Your Coding Agent at an Open Model

The harness you already trust — the one that reads your repo, edits files, runs your tests — was tuned for one vendor's model, but it was never married to it. Underneath the branding, a coding agent is a loop: gather context, take an action through a tool call, verify the result, repeat until the task is done. That loop is model-agnostic. It only needs a model that does two things well: follow instructions without drifting, and call tools in the right format. Everything else rides on top. So you can swap the engine without rebuilding the car.

Why the swap works at all

The agent talks to a model endpoint, and it reads that endpoint from environment variables. It doesn't hardcode a vendor. So you point those variables at a different provider — one serving an open model — set the credentials, name the model, and launch as usual. The harness sends the same tool definitions, the same system prompt, the same growing context, and it gets an answer back in the same shape. The harness never knows the difference.

That's the whole trick, and it's why "open vs. closed" isn't an all-or-nothing decision. The interface is a contract, not a marriage. Any model that speaks the contract can stand in.

The actual setup

Set three environment variables — a base URL, an API key, and a model name — then start the agent the way you always do:

export AGENT_BASE_URL="https://<provider>/v1"
export AGENT_API_KEY="<your-key>"
export AGENT_MODEL="<open-model-id>"
# launch your agent as usual

The exact variable names depend on your harness — check its docs for the ones it reads — but the shape is always the same: where to send requests, how to authenticate, which model to ask for.

If you use a router that already lists open models ready to go, it's often shorter still — a single base-URL change plus your router key. OpenRouter, for example, exposes an OpenAI-compatible endpoint at https://openrouter.ai/api/v1; point your client's base URL there, drop in your router key, and pick a model from its catalog (OpenRouter quickstart). One config change and you're running.

Test it on real work, not a toy

Here's the part people skip, and it's the part that matters. Do not judge the swap on a toy prompt like "write me a function." A one-shot answer tells you almost nothing about how the model behaves inside an agent.

Point it at an actual bug in an actual repo and watch it run a full loop: read the code, make a plan, edit several files, run the tests, read the failures, fix what broke, try again. What you're really checking is whether the model holds the thread across roughly ten tool calls without losing the plan. That only shows up on real, long-horizon work — the kind that runs for many minutes instead of answering in one shot. Models built specifically for long-horizon agent work are the ones that survive this; general-purpose chat models are where you'll watch the plan quietly fall apart around step ten.

The honest limits

Swapping models isn't free of tradeoffs, and pretending otherwise sets you up for surprises:

Safety and refusal behavior differs. A model from a different lab was tuned with different defaults. Requests one vendor allows, another may refuse — and vice versa. Expect the edges to feel different.
The co-designed experience is smoother. When a harness and a model are built by the same team, the little details line up in ways a swapped-in model doesn't perfectly reproduce. It's close, not identical.
The frontier still wins the hardest problems. On the trickiest, most novel architecture work, a frontier model earns its price. The everyday 90% — refactors, tests, glue code — is where an open model is closest to a cheat code.

Takeaway

The right posture isn't "switch everything" or "switch nothing." Use an open model for volume, keep a frontier model one config flag away for the hardest 10%, and treat the base URL as a dial you turn per task, not a religion. The harness is yours either way. The model underneath is just an environment variable.

Resources

Building an AI feature? Yeda AI designs, audits, and ships production LLM systems.

Talk to us · Read the blog