LLM Router: How Model Routing Cuts AI Coding Cost

Search “LLM router” and the first page is glossaries. They agree that a router is a layer that picks a model per request, borrow the same 85% cost-reduction figure from one 2024 paper, list eight products, and stop. None does the arithmetic for a real workload, none says how you would know the cheap model held, and none mentions the 2026 benchmark that found several routers do not beat a simple baseline.

An engineering leader asking the question has a narrower problem than the glossaries answer. Coding agents are the fastest-growing line on the AI bill, a session on a frontier model costs real money per hour, and most of what the agent does in that hour is reading files, running tests, and summarizing output. The question is whether the expensive model has to do all of it, and if not, who decides, and how you would catch a wrong decision before it reaches production.

This post answers that: what a router is, a ticket routed step by step at October 2026 list prices, then the part the glossaries skip, which is how YAGNI’s four lanes work, how a person stays in charge of the policy, and where the math breaks.

What is an LLM router?

An LLM router sits between an application and the models it can call and decides, per request, which model gets it. The goal is to pay the frontier price only for the requests that need frontier capability, and to send everything else to a model that is good enough and much cheaper. There are three ways to make that decision, and most products combine two of them.

Rules. The router reads something it can see without running a model: the task type, the prompt length, a tag the caller set. “Summaries go to the small model, anything with a diff goes to the large one.” Cheap, predictable, easy to audit, and only as good as the rules someone wrote.

Prediction. A trained classifier estimates, from the prompt, whether the cheap model will answer well enough, and routes to the strong model only when the estimate falls below a threshold. This is what RouteLLM trained on human preference data in 2024, and what Microsoft’s model router does in its Balanced mode, which picks the cheapest model within a 1% to 2% quality band of the best one for that prompt, as its docs describe as of October 2026. The threshold is the policy: a wider band saves more and misroutes more.

Cascade. Try the cheap model first, run a check on the answer, and escalate to the strong model only if the check fails. The check can be a verifier model, a test suite, or a schema. Cascades spend tokens twice on the hard cases and never pay the frontier price for the easy ones.

A router is not a gateway. A gateway is the proxy that holds the keys, enforces rate limits, logs spend by team, and fails over when a provider returns errors. Gateways often ship a router inside them, and a router without a gateway’s logging cannot be trusted, because you cannot see what it decided.

Why does model routing cut cost?

Because the price spread between a vendor’s models is far larger than the capability gap on most requests. Anthropic’s API prices as of October 2026, per million tokens:

Model	Input	Output	Cache read
Opus 5.5	$4	$20	$0.20
Sonnet 5.5	$2	$10	$0.10
Haiku 5.5 (prompts up to 100K tokens)	$0.10	$0.50	$0.01

Forty to one between the top of that table and the bottom, from one vendor, before open-weight models enter the picture. Anthropic’s own cost guidance for Claude Code says the same thing a router encodes: Sonnet handles most coding tasks well, reserve Opus for architectural decisions and multi-step reasoning, and give simple subagent tasks to Haiku. A developer who runs /model by hand is routing. A router does it on every step, without relying on a habit.

The research figure every glossary quotes is real but narrow. RouteLLM reported cost reductions of over 85% on MT Bench, 45% on MMLU, and 35% on GSM8K while keeping 95% of GPT-4’s quality, with the best routers matching two commercial routers at over 40% lower cost. Three benchmarks, three very different savings, which is the first honest lesson: the saving depends on how much of your traffic the cheap model can hold, and a chat benchmark says little about a coding agent’s step mix.

So here is the step mix instead.

What does routing a coding agent’s ticket actually look like?

A coding agent’s run is not one request. It is a sequence of steps with very different demands: read the ticket, map the repository, write a plan, make the edits, run the tests, review the diff, write the record. Two or three of those need the strongest model. The rest are reading and summarizing, where a small model is as accurate and forty times cheaper.

Here is one ticket, a change to a checkout service, costed two ways at Anthropic’s October 2026 list prices from the table above. Token counts are illustrative, in the range a mid-sized ticket produces before prompt caching, which lowers both columns. Each step is a sum over many requests, each well under 100K tokens, so the Haiku row above is priced at its small-prompt rate.

Step	Tokens in / out	Everything on Opus 5.5	Routed per step	Who did it on a YAGNI Team
Read the ticket, the Slack thread, and the Team’s decisions	60K / 2K	$0.28	$0.01 (Haiku)	Reeve, before planning
Map the repository: search, read the files the change touches	400K / 10K	$1.80	$0.05 (Haiku)	Reeve
Write the plan a person approves	150K / 8K	$0.76	$0.76 (Opus)	Reeve
Make the edits to a draft pull request	600K / 40K	$3.20	$1.60 (Sonnet)	Wright
Run the tests and read the output	200K / 5K	$0.90	$0.02 (Haiku)	Wright
Review the diff: intent, database, security, tests	250K / 10K	$1.20	$1.20 (Opus)	Proctor
Write the record of what was done and why	50K / 3K	$0.26	$0.01 (Haiku)	Harper
Ticket total	1.71M / 78K	$8.40	$3.65

A bit over half the bill, gone, with the plan and the review still on the strongest model in the catalog. Multiply by the tickets a Team carries in a month and that is the line a manager was asked to find, and it is routing inside one vendor’s menu, the floor rather than the ceiling.

Two things in that table matter more than the total. The expensive steps are the ones a person reads: the plan, which a person approves before a line is written, and the review, which a person reads before merging. The cheap steps are the ones nobody reads, where a small model doing a search or parsing a test log is checked by the next step, not by a human. That is the right place for a router to spend its savings budget, and it is why routing per step is safer than routing per session.

How does a router know the cheaper model held?

This is the question the glossaries do not ask, and it is the one that decides whether routing is a saving or a slow leak. A router’s own accuracy claim is not an answer. LLMRouterBench, a January 2026 benchmark that put dozens of routing methods under one evaluation, found that many perform about the same and that several, including commercial routers, fail to reliably outperform a simple baseline. The routing decision is a prediction, and predictions are wrong some of the time.

What makes a misroute cheap is a check that catches it before anyone pays for it twice, in the order that costs least:

1. The next step catches it. A search that missed a file shows up when the plan cites the wrong module; a test log parsed wrong shows up when the tests still fail. Steps that feed other steps are self-checking, which is why they are safe to route cheap.
2. A gate catches it. The plan goes to a person before any code is written. A plan built on a misread ticket costs one send-back, not a build. On a YAGNI Team that gate is Reeve’s plan, which a person approves, and the Agent reads the Team’s decision ledger before it plans, so fewer plans are wrong in the first place. Context engineering for coding agents covers why that lowers retries more than any model choice.
3. The review catches it. Proctor, the Reviewer, runs its checks on the pull request and posts one review a person can read, so an edit a cheaper lane got wrong is a finding, not a merge.
4. QA catches it. Fletcher, in beta, boots the repository environment in a sandbox, walks the change in a browser, and attaches video and screenshots.
5. A person merges. Always. The router never has the last word.

The escalation rule follows from this. A step whose check fails goes back on a stronger lane, and the record shows that it did. A router that cannot escalate is a cost cap, not a router.

The second thing a coding agent needs from a router is to respect the prompt cache. An agent’s requests carry the whole conversation so far, and after the first turn most of that is served from the cache at a twentieth of the input price. Switching models in the middle of a run throws the cache away, which is why Microsoft’s docs note that caching applies only when the same model handles consecutive turns. Route at step boundaries, where the context changes anyway, and the cache survives.

How do YAGNI’s four lanes work?

YAGNI does not publish a model list. It publishes four lanes, Efficient, Standard, Advanced, and Peak, each with a per-million-token price on one rate card, locked for twelve months per contract while the models underneath rotate. As better models land, the lanes are upgraded and the rate does not move. The models are vetted US-hosted open-weight models, and the floor across the card is 60% or more under comparable frontier API rates. The rate card itself is shared on a call, not published; that floor is the only public number.

The router places each step of a run on the cheapest lane that holds quality, and a hard step goes to a stronger lane when the cheaper one would not hold. So the ticket above runs the same way on YAGNI, with the read and test steps on Efficient and the plan and review on the stronger lanes, except that every lane sits under the comparable frontier rate rather than inside one vendor’s catalog. A step is billed at the lane the router placed it on. If the model on that lane is unavailable and the call is rescued by a higher lane, the customer still pays the placed lane; a model being sick is YAGNI’s cost, not yours.

Three groups of users ride the same router:

• Developers on YAGNI Code, the CLI and desktop coding agent, where the router sizes each piece of work before it runs, so mapping a repository never pays what a hard refactor pays.
• Developers on Claude Code or Codex through YAGNI’s wire adapter. One command, yagni connect claude-code, and the same session runs on YAGNI’s lanes with the same workflow, the same skills, and the same MCP servers. How much Claude Code costs walks through that arithmetic.
• The named Agents on a Team. Bailey proposes the next ticket from the backlog in Jira or Linear, Reeve plans it, Wright builds it to a draft pull request, Proctor reviews it, Fletcher tests it in beta, and Harper posts what shipped to Slack with the receipts attached. Every one of those runs is metered on the same card as the developers.

On YAGNI the router sits under a Team that is managed like an engineering team, so the record a manager already uses to run the Team is the record that shows what the router did.

Who owns the routing policy?

A person, with a record. Routing nobody can inspect is a cost cap somebody set once and forgot, and the first sign of a misroute is a reviewer’s afternoon. Three things keep it owned on YAGNI:

The record shows the lane. Every run on a ticket’s case file shows what the workspace was charged for it, and the Usage page shows spend per developer and per Agent, with each Agent’s spend broken out by the job that spent it. A developer or an Agent whose steps landed on different lanes shows up on those lanes, exactly like a developer who switched models by hand. “Proctor’s reviews on the checkout repository ran mostly on Advanced this month” is a sentence a manager can read, not a log line to reconstruct.

The brake is a person’s. A daily spend ceiling in Workspace settings stops the meter; a Team’s Pause stops every job on the Team at once. The router can lower a bill; it cannot raise one past the ceiling a person set.

The gates are a person’s. The three clicks of a supervised ticket, accept the proposal, approve the plan, merge, do not change with the lane. A line moves to autonomous only by a person’s explicit act, with the record beside it, and merges never leave a person. The router decides which model does the work. It never decides whether the work ships.

Where does the math break?

Honest arithmetic names its own failure modes.

• Hard tickets favor the frontier. On a task the cheap lane cannot hold, a cascade pays twice and a predictor misroutes. A migration touching three services is an Advanced or Peak ticket from the first step, and a router that forces it cheap makes it dearer. When the task mix is hard enough, the frontier premium is worth paying, and we say so on the call rather than pretend the floor is free.
• Router overhead is real. A classifier call on every step is not free, and a cascade’s failed first attempt is not free. The saving in the table above assumes the routing decision costs a rounding error, which holds for rules and cheap classifiers and does not hold for a router that uses a frontier model to decide.
• Retries eat savings. The cheapest step is the one that does not run twice. An agent that guesses at why a ticket exists takes the plan, build, send-back, rebuild loop, and no lane is cheap enough to make four runs cost less than one. Grounding, the decision ledger and the connected tools an Agent reads before it plans, moves the bill more than the lane does.
• Token price is not task cost. A cheaper model that uses twice the tokens and needs a second attempt costs more than the expensive one that finished. The per-ticket total on the case file is the number to manage, not the per-million price on the card.

Start with the step, not the model

The glossaries frame routing as a model-choice problem. For an engineering org it is a management problem. A ticket is a sequence of steps, two of which a person reads and the rest of which a check reads, and the expensive model belongs on the two. A router that places each step on the cheapest lane that holds quality, escalates when it does not, and leaves a record of which lane it chose is the mechanism. The gates around it, a plan a person approves, a review a person can read, a merge a person clicks, are what make the saving safe.

That is how a YAGNI Team runs, and how YAGNI Code runs for the developer at the terminal, on one rate card at 60% or more under comparable frontier API rates. If you want the ticket arithmetic run on your team’s step mix, book 30 minutes and bring last month’s bill. The bigger picture, how a Team carries work from proposal to merged pull request under a person’s review, is in Agent Teams for software development, and the offer is on the home page.