Hybrid: local first, the cloud only when it is worth it

Level AdvancedEstimated time 1 hRead first One gateway for all your models, and three ways a local model freezes your agents, A tiny local judge for the small decisionsA word you don't know? Glossary

Local models handle most of the work; some tasks still need a frontier model. Paying a subscription for everything wastes its quota on what a local model does well; refusing the cloud entirely leaves the hard tasks undone. Our answer is a small escalation queue: each task goes only as far up as it needs.

A task climbs only as high as it needs. Local models first; if they cannot, the cheaper hosted agent; the most expensive one only when the task demands it or you pinned it. Each hosted agent has a daily cap and a cage, and nothing climbs without your confirmation on the phone.

1. A queue, not a reflex

Agents do not call a hosted model directly. When a local agent cannot finish a task, it files it in an escalation queue with what it already tried. The queue decides who takes it: by default the cheaper hosted agent, the frontier one only if the cheaper one is out of quota or does not answer, or if you pinned the task to it (hard problems, critical changes, final reviews).

2. Caps and cages

A daily cap per hosted agent (ours: 30 tasks a day for the cheaper one, 10 for the frontier one): a loop cannot burn a month of quota in a night.
An allow-list of tools for the agent that supports one, with no sudo, no recursive delete, no killing by name.
A sandbox for the agent that has no allow-list: a read-only system, write access only to its working folder.

3. You confirm

Nothing climbs to a hosted agent without a card on your phone: the task, the agent chosen, a yes or a no. Large projects are split into ordered tasks, with a final review by the strongest model.

Traps we paid for

Start tasks in order. Our first project runner started every task at once: a "review then deploy" task ran on code that had not been written yet. Dependencies are part of the plan.

Private data stays local: route by the data's class, not only by difficulty (see the decision cascade in Homelab 14).

A context-compression proxy in front of a hosted model saved nothing on prompts under about 100,000 tokens, and cost a little more. Measure before adding a layer.

Found this useful? Tell us what to measure next.