The deterministic doctrine: the model proposes, the code decides
Ask a language model the same question twice and you may get two answers. That is a gift when you want ideas, wording, a plan. It is a disaster when you want a verdict: does this test pass, is this file valid, may this command run. Our whole stack rests on one separation: the model proposes, the code decides. Everything that has one right answer is done by a program that answers the same way every time.
How these figures were obtained. Measured on our own machines (see the protocol), usually a single run, with speeds as reported by the server itself; we do not publish raw logs. Treat them as orders of magnitude on this hardware, not as guarantees on yours. Figures from outside are labelled with their source.
What you get. Small local models that deliver real work, because they are never asked to do what code does better. Verdicts you can replay, audit and trust. And a bill that shrinks, because a deterministic tool costs no thinking and almost no context.
1. Two kinds of questions
| Many right answers: a model | One right answer: code |
|---|---|
| split a project into stories | does this split respect the schema? |
| write the shot list of a film | does every shot use a framing from the list? |
| choose which tool fits a request | are the arguments of that call valid? |
| word the reply to a customer | does the reply contain a credential? |
| propose a fix | do the tests pass now? |
The left column is creative, so a model is the right tool. The right column is a check, so a model is the wrong tool: it would answer "probably" where you need "yes" or "no".
2. The doctrine, layer by layer
- Tools: a tool reads JSON, writes JSON, and its exit code carries the meaning: 0 success, 1 a negative verdict, 2 a refusal, 3 and above a technical error, never a verdict. Replayed three times, it must give the same bytes three times (see fh-tools).
- Agents: the model designates a function from a closed table; the code checks the arguments and runs it (see the workstation assistant).
- Projects: the model proposes the split, a linter judges it, you approve; scenarios run as plain tests (see BMAD).
- Routing: deciding which model answers is code, not a sentence in a prompt (see the local judge).
- Authority: the account that reasons holds no secret; a daemon accepts only a name from a closed list (see agents and authority).
- Creation: the model writes the shot list, the code refuses it before a card lights up (see a sentence becomes a film).
3. The numbers that convinced us
- Told in its instructions to ask the judge, an agent did it 0 times out of 3. Routed by code, every time.
- Asked "why does this fail?", a 27B model scored 2.75/5 in 9 minutes; handed the facts by a tool, 5/5 in 72 seconds.
- Shot lists checked by code before rendering: 4 valid out of 4 on the first try; a bad list found after rendering costs two hours.
- Coding agents deliver merged pull requests with six tools, 2,182 tokens, where a catalogue of 58 tools weighed 19,157 tokens at every turn.
4. A good tool is often worth more than a skill
A skill teaches the model a procedure; the model still has to read it, understand it and apply it, with its usual variance. A tool is the procedure: the model only has to call it.
| Skill | Tool | |
|---|---|---|
| cost while idle | its description | its description (120 characters at most here) |
| cost when used | the whole procedure, read into the context | one call, one JSON answer |
| result | as good as the model's reading | the same every time, testable from outside |
| works with a small model | sometimes | yes: it only has to choose and fill the arguments |
So: when a procedure has a single right outcome, write the tool. Keep skills for what needs judgement: wording, design, splitting, review. The smaller your model, the more this matters.
5. A bug becomes code, never a note
A rule written in a guide in the morning was broken in the evening by the person who wrote it. Notes are read and forgotten; code runs every time. When something fails once, the fix is a check, a lock, a refusal, a test, ideally one you have seen go red (see mutation testing). What cannot become code becomes a journal entry you can search.
Traps
A confidence score measures how sure a model is, never whether it is right. Two options within 0.05 of each other compete: ask, do not pick in silence.
Do not bend the contract for convenience. A "tool" that calls a model is not deterministic; say so in its name, or every other guarantee is worth less.
Found this useful? Tell us what to measure next.