Routing is a session problem, not a request problem

Databricks reports that routing each task to the right model matches frontier quality at 30%+ lower cost (post). Prefex saves us 15–20% beyond native prompt caching (16% over the last 30 days), but that comes from caching, compaction and everything else together; ours is not 30% from routing alone. We wanted to know whether that gap is our caching making excuses for us, or a real difference in the work. So we measured 30 days of our own coding-agent traffic, and the answer is the shape of the work. Routing pays where work arrives as tasks; most of our spend arrives as a handful of long, reused sessions.

What Databricks got right

They route once per task, at the start, and never switch mid-session, because “at scale, costs are dominated by cache hit rate.” A session that starts on a cheaper model builds its own cache from turn 1, so the saving is the full price gap with no cache penalty. We agree, and our measurements say the same: cached reads are about 96% of our input tokens.

Their companion benchmark adds two points worth repeating (post):

• Price per token misleads. Sonnet 5 is 1.7x cheaper per token than Opus 4.8 but cost more per task ($2.09 vs $1.94), because it used 1.9x the tokens.
• The harness matters as much as the model. The same model at the same effort cost over 2x more per task in some harnesses, mostly from how much context each turn sends.

They also name the limit themselves: opening prompts are rarely precise, and sessions get reused.

Where our spend actually sits

Over 30 days, main-loop Claude Code cost $4,497 across 7,458 sessions. Thirty sessions of 100+ turns carried 81% of it. The 1,322 short sessions a session-start router could pick for cost $152 between them.

Those long sessions average about 700 turns each. They are the kind of work an engineer keeps open all day and points at one task after another.

Why per-task routing doesn't transfer

A pick at session start can only reach the Opus sessions under 100 turns, about $280 of our month. The long sessions start with one vague message and then run for hundreds of turns across many tasks, so committing them to a cheaper model on turn 1 is a guess. Inside them, Opus spent $2,961 over 29,853 turns at an average of 228k tokens of context per turn. The split was 45% re-reading cached context, 25% cache writes and 14% output. The cost is a huge context re-read every turn, priced at Opus rates.

Switching models partway is less costly than we assumed, but it isn't free. When each model keeps a warm cache of the same conversation, a switch writes only what was added since that model's last turn, not the whole prefix:

The switch back is the costly turn: Opus re-writes everything that happened while Sonnet answered. In these sessions the switches cost $2–4 in extra writes and lost reads, against about $10 saved. A small win, and too small to explain a 35% gap either way.

The shape that could work: a group of models, each with its own cache

If routing pays per task, the move for long sessions is to make tasks inside them. Picture one model name the client talks to, with several model sessions behind it, each keeping its own warm cache. Whether that saves money depends almost entirely on what each model has to carry.

• Every model carries the whole conversation. A cheaper turn saves the price gap on the 228k re-read. But each switch back re-writes the gap at the expensive model's rate, and every extra model's cache has to be kept warm. The answer turns on how often you switch and how long each run lasts.
• One model coordinates and the others get a short brief. A worker with a small context is cheap whatever model it runs, and the coordinator's turns get fewer. This is the sub-agent pattern, and it's where we have the most room: our sub-agents cost $1,160 over 18k turns with an average context of 143k, because forks inherit the whole conversation.

Two smaller findings point the same way. Claude Code's own utility calls (titles, summaries) cost $276 a month, each carrying about 148k tokens of context to answer about 25 tokens. And sessions people deliberately ran on Sonnet were short and cheap, which says little about how Sonnet would do on a 700-turn session.

What we're doing next

We aren't calling this settled by cache economics. Next we're measuring it:

1. A turn-level simulator over the long sessions. It replays each real turn sequence under a routing policy, keeps a separate cache per model (reads, new writes, expiry) and prices every turn. Policies: all Opus; a session-start pick; Sonnet for low-output turns; alternating runs on two models; Haiku for utility calls.
2. Shadow decisions on live traffic. Log what a session-level and a task-level router would have picked, beside what the session actually cost, without changing anything.
3. Quality from outcomes, not a judge. Like Databricks, we won't let a model grade answers. We'll use what happened to the code: did it land, was it reverted, did it survive.