Strata is fast. Keeping it warm is the other half.
We ran a 125-billion-parameter model on one RTX 3090 at about 140 tokens a second, using Strata, an open-source (MIT) engine built on llama.cpp. Then we put PromptForce's cache control in front of it. On a 20,000-token conversation, the next answer started in 0.64 seconds instead of 8.1.
What we measured
Strata runs Qwen3.8-Flash-Next, a 125B mixture-of-experts model, on a 24 GB card by keeping the experts it isn't using in system RAM. We compared it with the two engines we already run, on the same machine and the same prompts. Figures are tokens per second: how fast each engine reads the prompt and writes the answer, at 4k- and 28k-token prompts.
| Engine and model | Read 4k | Read 28k | Write 4k | Write 28k |
|---|---|---|---|---|
| vLLM, Qwen3.8-27B dense (4-bit) | 1,215 | 820 | 103 | 81 |
| llama.cpp, webAI Air 26B-A4B MoE | 2,700 | 3,400 | 123 | 101 |
| Strata, Flash-Next 125B MoE (Q2_0) | 1,630 | 2,720 | 141 | 139 |
A model almost five times larger wrote answers faster than the dense 27B, and its output speed held as the prompt grew. Strata deserves the praise it's getting.
What tokens per second leaves out
A coding agent re-sends the whole conversation on every turn: the system prompt, the files it read, and everything said so far. At 20,000 tokens that is the normal case. Output speed only matters once the model has read all of that again.
If the engine still holds the conversation in its cache, the re-read is nearly free. If anything has pushed it out, such as another chat, a background job or a few minutes of idle, the engine reads it all again from scratch. On our run that took 8.1 seconds before the first word appeared, however fast the words came after it.
Strata already keeps a few recent conversations on its own. It doesn't let you say which one matters, and it doesn't keep one through a busy spell.
What PromptForce adds: a say in what stays cached
prefex sits between your coding tool and the model. Its cache control for local engines, kvzones, lets a session mark the part of its prompt that won't change and hold a lease on it. A small service beside the engine keeps that part warm and parks it on disk when memory runs short. Every answer then reports how much of the prompt it reused.
We put Strata behind it without changing either side: kvzones already speaks the llama.cpp interface Strata provides. Then we ran two sessions over the same code, one with a lease and one without, and sent five other sessions plus 20 seconds of idle between their turns:
| After other traffic and 20 s idle | Prompt reused | Time to first word |
|---|---|---|
| Session with a lease | 20,250 of 20,271 tokens | 0.64 s |
| Session without one | 0 of 20,298 tokens | 8.1 s |
The fast engine still decides how quickly the answer is written. The lease decides whether the answer starts at once or after the whole conversation is read again: 13 times sooner here. The same checks pass on vLLM and llama.cpp, so one setting covers all three engines.
What to keep in mind
- One session at a time. Strata serves one conversation at once, so it holds one session warm. It suits one developer on one GPU, not a shared team server.
- Output speed depends on the text. Strata drafts several tokens ahead and checks them. Code review is easy text to guess, so expect less on harder output.
- Saving to disk needs a manual start. Strata's launcher doesn't pass its slot-save option through, so we started its server directly.
- One machine, one run. These are single measurements on an RTX 3090 with 62 GB of RAM, not a benchmark suite.
kvzones is in preview in prefex. If you run Strata for a coding agent and want to try it, tell us.
Sources
- Strata on GitHub, version 0.2.11, engine 0.1.57
- Model: Qwen3.8-Flash-Next GSQ-RCO Q2_0 GGUF from ISTA-DASLab
- Measurements: PromptForce, 2026-11-06, RTX 3090 24 GB