Free cloud models behind your gateway, and routes that say who answers

Level IntermediateEstimated time 1 hRead first One gateway for all your models, and three ways a local model freezes your agentsA word you don't know? Glossary

Your cards are the first choice: private, free once bought, and as fast as you made them. But a card running a game, a long job or a reboot answers nothing. Several providers give real models away for free, within daily or per-minute quotas. Put them behind the same gateway, and group them into routes: one name that tries a list of models in order. The rule that keeps it honest: only a name that starts with auto- may have another model answer.

What you get. 34 cloud models added to the gateway, next to the local slots, with a total budget of 0 €. Three routes: auto-local-first (the cards, then free APIs), auto-gratuit (free APIs only) and auto-difficile (the strongest free models, then offered credits last). Every client that already talks to the gateway sees them without changing anything.

A request for a route name tries its members in the listed order: a local slot that is not loaded refuses at once, and the next member, a free API, answers. A request for a model's own name never falls back: it gets the error.

A bit of history

Free tiers for language models appeared in 2024, when new inference companies (Groq with its own LPU chips, then Cerebras with wafer-sized ones) needed developers to try them. The large labs followed with free keys for their smaller models. The landscape moves fast, and in both directions. In 2026 alone, GitHub Models closed on 30 July. The free Gemini CLI gave way to the Antigravity CLI on 18 June. The free Qwen OAuth access ended on 15 April. And since 23 March, a Google AI Studio project linked to an empty prepaid billing account refuses every call, free models included. A free source is a loan, not a foundation: that is why the cards stay first.

1. What is allowed, and what is not

Official offers only: a key from the provider's own console, used through its documented API.
One account per provider. Rotating several accounts to multiply a free quota breaks the terms of every one of them.
No reverse-engineered aggregator that replays a web chat session as an API: it breaks the terms and it breaks without warning.
Read the data clause. Some free tiers may use your requests to train their models (Mistral's free "Experiment" plan says so). Send no secret, no client code, nothing personal through them.

2. The sources we wired, and what they did on the day

Provider	Account	Tested on 21 October 2026
Groq	email, no phone	3 models answer in about a second
Cerebras	email, no phone	2 models answer
Google AI Studio	Google account	Flash-Lite answers; Flash answered "overloaded" (503)
Cloudflare Workers AI	email; the address carries the account id	3 models answer
OpenRouter	email or GitHub	4 `:free` models answer; without credit, 50 requests a day shared by all of them
Cohere	email	2 models answer
NVIDIA NIM	phone required	2 models answer; others time out or are not found, but stay declared
Mistral	phone required	Codestral answers; one request per second on the free plan

Declare a model even when it fails today. A free model that is overloaded or not yet open comes back; a route simply skips it until it does. Removing it by script on the first failure turns a bad hour into a missing model.

3. No key in a file you could share

A key goes from the provider's console to your secrets manager, from there to one environment file read by the gateway, and nowhere else. The gateway configuration names the variable, never the value:

# the key goes straight from the clipboard into the secrets manager
wl-paste | tr -d '\n\r ' | your-vault put GROQ_API_KEY
wl-copy --clear

# the gateway's environment file: mode 0600, loaded at start
#   GROQ_API_KEY='…'

# in the generated configuration, only a reference:
api_key: "os.environ/GROQ_API_KEY"

Remote models live in a map next to the one that lists the local slots, one line per model:

# remote.map — name|api_base|upstream_model|max_input_tokens|api_key
groq-gpt-oss-120b|https://api.groq.com/openai/v1|openai/gpt-oss-120b|131072|os.environ/GROQ_API_KEY
cf-gpt-oss-120b|https://api.cloudflare.com/client/v4/accounts/YOUR-ACCOUNT-ID/ai/v1|@cf/openai/gpt-oss-120b|131072|os.environ/CLOUDFLARE_AI_TOKEN

The generator refuses a remote named like a local slot: two models behind one name is exactly the confusion the gateway exists to prevent.

4. Routes: several models behind one name, and the name says so

A third map lists the routes. Each line is a name and its members, in the order they should be tried:

# routes.map — route|member,member,...
auto-local-first|local-27b-card1,local-9b-card1,groq-gpt-oss-120b,cerebras-gpt-oss-120b,gemini-flash-lite
auto-gratuit|groq-gpt-oss-120b,cerebras-gpt-oss-120b,gemini-flash-lite,mistral-codestral
auto-difficile|nv-glm-5.3,or-nemotron-3-ultra,umans-glm-5.3

The generator turns each route into one gateway entry (a copy of its first member) and one fallback list, and nothing else falls back:

router_settings:
  # Only routes fall back, and only to the models they list.
  fallbacks:
    - "auto-gratuit": ["cerebras-gpt-oss-120b", "gemini-flash-lite", "mistral-codestral"]

It refuses the whole configuration, rather than half of it, when a route does not start with auto-, names an unknown member, lists one twice, or reuses an existing model name. Without a routes file, its output is byte-for-byte what it was before.

Order a route by what you are willing to spend. Offered credits that do not renew go last: they serve only when every free model has failed.

5. Prove the fallback, do not assume it

LiteLLM documents a mock_testing_fallbacks flag that should simulate a failure. On our version (1.92) it changed nothing: the first member answered every time. So we built a real failure instead, without touching the running gateway:

Generate a small configuration with one route: first member, a local slot that is not loaded; second member, a free API.
Start a second gateway from it on a spare port, inside the same container, under timeout 120 so it stops by itself.
Call the route: the free API answered, and the response header said x-litellm-attempted-fallbacks: 1.
Call the unloaded slot by its own name: a connection error, "No fallback model group found". That is the second half of the proof.

On the real gateway, the header x-litellm-model-api-base tells you which member answered each request: your card, or which provider.

Traps

The empty configuration. Over a non-interactive SSH session, ~/.local/bin is often not in the PATH. A line such as gen > tmp || true; mv tmp config.yaml then installs an empty file, and the gateway serves no model at all. Call the generator by its full path, and replace the configuration only after checking it: it is not empty, it holds the expected number of models, and no api_key holds anything but a reference.

Google answers 402, "prepayment credits are depleted", on a free model. The project is linked to a prepaid billing account at zero. Detach the billing from that project in AI Studio, create a new key, and the free tier works again.

Never print part of a key "to check it". Three characters in a terminal are three characters in a log. Check a prefix with a yes/no test, never by displaying it.

Free is slow when it is busy. A strong free model took 115 seconds for a one-word answer through a shared queue. Give agents a timeout that matches, or keep the slow ones at the end of their route.

Sources

Found this useful? Tell us what to measure next.