billion-context: a context-compression proxy, measured honestly
billion-context is an open-source proxy: your agent talks to it instead of its model, and it rewrites the conversation on the way, giving the model tools to compress and recall what is old. The promise is long sessions without hitting the window. We do not adopt a tool on a promise: we measured it on the three agents we use.
How these figures were obtained. Measured on our own machines (see the protocol), usually a single run, with speeds as reported by the server itself; we do not publish raw logs. Treat them as orders of magnitude on this hardware, not as guarantees on yours. Figures from outside are labelled with their source.
What you get. A clear answer before you spend an evening on it: no gain under about 100,000 tokens of context, a real extra cost in some cases, and the safe way to install, plug in and remove it if you want to try on long sessions.
1. Install it tightly
- Pin the version and install without running package scripts (
npm install --ignore-scripts): its only dependency is a validation library. - Run it as a user service bound to the machine itself, with
NoNewPrivilegesandProtectSystem=strict. - Turn off automatic updates and the "check for news" calls in its configuration.
2. Plug it in, one agent at a time
- Claude Code: point
ANTHROPIC_BASE_URLat the proxy, with a direct fallback if it does not answer. Works with a subscription; no token appears in its log. - Kimi Code: changing the address does not work, because Kimi stores its login under a key derived from that address, and loses it. Use the proxy's native plugin instead: it adds a separate provider that keeps the same login key.
- OpenFox: one switch, the model address it uses.
3. What we measured
| Test | Direct | Through the proxy |
|---|---|---|
| Claude, 40 files read in parallel | $0.19 | $0.62: the proxy changes the start of the prompt, so the model's cache starts cold |
| Kimi, a chain of 30 files | 132 s, correct | 102 s, correct, but 13 % more data sent and no compression triggered in 31 requests |
| OpenFox, a chain of 40 files on a local 27B | stopped at 15 files | stopped at 13 files, 60 % more prompt reading |
Verdict: under about 100,000 tokens, nothing to gain and a small cost (its own tool descriptions add about 17 KB to each request). Its expected effect is on small local windows, 32k to 64k, on very long sessions: still to be shown. We put every agent back on its direct route.
Traps
A proxy that rewrites the prompt breaks the provider's prompt cache. On a subscription billed by tokens, that alone can triple a bill.
Do not copy an agent's login token to make a proxy work: tokens that renew themselves can invalidate the original. Use a mechanism that shares the existing login.
Keep a one-line way back: an environment variable or plugin remove. Ours were used the same evening.
Sources
Found this useful? Tell us what to measure next.