Stacking free tiers: the quota warden
A frontier subscription (Kimi, Claude…) drains in cycles: a few hours of intense work, then a wall until the reset. Meanwhile, eight free providers (Groq, Cerebras, Gemini, Cloudflare, OpenRouter, Cohere, NVIDIA NIM, Mistral) each offer a few hundred or thousand requests a day. None replaces a subscription. But added up, they comfortably cover routing, validation, small tasks and part of the coding — given three ingredients: a bit of automation, thread teleportation, and provider handoff.
1. The sum nobody does by hand
Switching providers by hand means copying context, finding the right key, praying the quota holds. Nobody does that ten times a day. Our answer is three pieces:
- A router that knows everyone. Every free provider is declared in a single gateway (for us: LiteLLM, ~35 free models behind one endpoint). Applications only talk to the gateway; they don't even know who answers.
- Thread teleportation. When a model runs dry, the whole conversation — context, memory, files — migrates to the next provider losing nothing. That is swarmfox's "teleport" feature: the thread is a suitcase, not a mural. Provider handoff becomes a non-event.
- A quota warden that watches every call and learns each model's state:
ok,cooldown,exhausted,down. A 429 with a reset time → the model is greyed out until that hour. A bare 429 → fifteen minutes of cooldown. Three server errors in a row →down. A forced call that succeeds → back took; ground truth always wins.
2. Routes: cascading escalation
Models are arranged into named routes that cascade on their own:
auto-local-first— local cards first (no quota, just VRAM), then the free tiers.auto-gratuit— free only, fastest first.auto-difficile— the strongest free models, with non-renewable credits as the very last resort.
When Groq answers 429, the cascade tries Cerebras, then Gemini, then Mistral… The request lands somewhere, and the warden learned something. Next time, the exhausted model is not even tried: it stays greyed out until its reset.
3. What it actually changes
Before: empty frontier quota = no work until the next cycle. After: the background work (routing, review, small tasks, writing) keeps going on the stacked free tiers, and frontier quota is reserved for what truly justifies it — heavy code, final review, hard decisions. The subscription does not change; what we ask of it does. A frontier quota cycle no longer bounds the day, only the peak tasks.
House rules
- No silent spending. Non-renewable paid credits never sit in an automatic cascade: only in a route you call by name.
- A named model never lies. Asking for
groq-gpt-oss-120breturns that model or an error. Onlyauto-*routes cascade. - Greying is visible. The UI asks the warden and greys the model for the length of its regeneration: you see the swarm breathe instead of discovering the wall by hitting it.
Traps we paid for
Free tiers promise nothing: variable latency, weaker models on complex code, quotas that change without notice. Treat them as a bonus, never as a foundation.
A supervision callback must be invisible when it works and silent when it breaks. Our first version failed the user's request when the warden was unreachable — exactly the opposite of the goal. Everything sits in absolute try/except, and async hooks are mandatory: the proxy never calls the synchronous ones.
On the proxy, the model name in the logs is the upstream id (openai/gpt-oss-20b), not your deployment name. The warden must read model_group, or it greys a name your UI does not know.
Found this useful? Tell us what to measure next.