The lab notebook: what we tried on the infrastructure, what we kept
A homelab is a series of experiments. Most write-ups only show what worked. Here is the whole notebook: what we tried, the measure that settled it, and what we did with it. Each line links to the full tutorial when there is one.
How these figures were obtained. Measured on our own machines (see the protocol), usually a single run, with speeds as reported by the server itself; we do not publish raw logs. Treat them as orders of magnitude on this hardware, not as guarantees on yours. Figures from outside are labelled with their source.
What you get. Months of trial and error in one page. Before you try something, check whether it is here: the number that decided for us may decide for you.
Kept
| Experiment | The deciding number |
|---|---|
| A 177B MoE model on two consumer cards (tutorial) | 55.6 tokens/s, 2.1× faster once the weights were placed right |
| Speculative decoding, MTP (tutorial) | 65.7 tokens/s on a 27B on one card |
| KVMem for long agent contexts (tutorial) | 48/48 recall, flat past 100k tokens, −4.9 GB VRAM |
| A 0.8B judge on the processor, then a 9B (tutorial) | 0.87 to 0.91 on real e-mails, 0.05 s per decision |
| A trained intent classifier instead of keyword matching | 84.0 % against 63.7 %, and 72.6 % for "always the default" |
| Regenerating the GPU device description at boot (tutorial) | the cause removed at its source, at every boot |
Rejected
| Experiment | Why |
|---|---|
| Compressing the V cache to q4_0 (tutorial) | 21× slower prompt reading for 1.8 GB saved |
| Model weights loaded resident instead of mapped | +3 % speed, then two machine freezes: 47.6 GB of RAM that could not be freed |
| A probability calibration that scored better on paper | 86.9 % instead of 84.0 %, but it never predicted the "hard" class again |
| A second small judge, aligned rather than trained | 0.42 to 0.55 on the same e-mails: deleted |
| Telling an agent, in its prompt, to ask the judge | it did so 0 times out of 3; routing went into code |
Paused
| Experiment | Where it stopped |
|---|---|
| Full-duplex voice (talking over each other, like a phone call) | a 35B model plus speech models: too heavy next to everything else on two cards. Making the house voice more responsive comes first |
| An automatic switch between KVMem and plain llama.cpp | switching costs 17.9 s; two presets do the job |
| Steering a local model token by token with the language server | no token masking inside llama-server; the workaround, cutting the request into segments, costs only 0.7 ms each. Feasibility proven, not built yet |
The method
- One variable at a time, three runs, median and range.
- Measure where it is used: an agent in a real session, not a
curl. - Break it on purpose to see the test go red (see mutation testing).
- Write the verdict down, with its number, the same day.
Found this useful? Tell us what to measure next.