Our findings: every tip and measured fact, one line each

This page is the short version of everything: the discoveries, the tips and the benchmark results we collected since we started, one line each. Nothing here is copied from a forum; the lines come from our own machines (method in the note below), and link to the tutorial that tells the story.

How these figures were obtained. Measured on our own machines (see the protocol), usually a single run, with speeds as reported by the server itself; we do not publish raw logs. Treat them as orders of magnitude on this hardware, not as guarantees on yours. Figures from outside are labelled with their source.

What you get. Months of evenings in fifteen minutes of reading. Skim it before your next upgrade, your next model, or your next "why is it slow?".

Graphics cards and PCIe

• A card can run at x4 instead of x16 after a reboot, silently. "Auto" in the BIOS is x16; a power cycle brought it back (tutorial).
• Find a card by its identity, never by its bus address: addresses move between boots (tutorial).
• A second card on OCuLink tops out at x4, 63 Gbit/s, and it is enough for inference (tutorial).
• After a driver update or a reboot, a container can see the card and still run on the CPU: regenerate the device description at every boot (tutorial).
• Without CUDA_DEVICE_ORDER=PCI_BUS_ID, "card 1" for CUDA may not be "card 1" for nvidia-smi (tutorial).

Models and speed

• Compressing the V cache to q4_0 made prompt reading 21 times slower, silently. Use q8_0 for both K and V (tutorial).
• A 177B mixture-of-experts model went from 26.1 to 55.6 tokens per second once its weights were placed right (tutorial).
• Speculative decoding (MTP): 65.7 tokens per second on a 27B on one card (tutorial).
• KVMem: 48/48 recall at 130k tokens, flat speed past 100k, 4.9 GB of VRAM saved (tutorial).
• Thinking left on burned 2,500 tokens without a line of code; off, the same code in 2.5 s (tutorial).
• llama-server ignores enable_thinking at the root of a request; use chat_template_kwargs (tutorial).
• Model weights held resident instead of mapped: +3 % speed, then two machine freezes (notebook).
• Watch memory pressure, not "available memory": it said all was well three minutes before a freeze (tutorial).

Agents

• Small and well-framed beats big and vague: the 9B delivered clean pull requests where the 27B produced nothing (tutorial).
• Declare tools as Python signatures: in JSON Schema, models chose well but did not fill the values (tutorial).
• 98 % of an agent's bill is the same 30,939 tokens of instructions and tools, sent at every turn (tutorial).
• Told to ask the judge in its prompt, an agent did it 0 times out of 3; routed by code, every time (tutorial).
• Handed the facts, a 27B found 5 faults out of 5 in 72 s, against 2.75 in 9 minutes without (tutorial).
• A 0.8B judge on the processor: 0.05 s per decision, never wrong above 0.8 confidence on our e-mails (tutorial).
• A context proxy saved nothing under 100k tokens and once tripled the bill by breaking the prompt cache (tutorial).
• Never kill by name pattern over SSH: the pattern matches your own shell (tutorial).

Creation

• Four-step distillation rendered a video shot 6.8 times faster, and the picture was better (tutorial).
• A video model renders what follows the action: write "still" or a present participle (tutorial).
• Three minutes of music in 27 seconds; store it as Opus, 3 MB instead of 69 (tutorial).

Homelab and workstation

• With Docker, fail2ban bans must go in the DOCKER-USER chain, or they block nothing (tutorial).
• A service published on the internet is mostly visited by scanners: publish only what needs it (tutorial).
• ExecCondition treats exit code 255 as a failure, and 255 is what ssh returns when the host is away (tutorial).
• "100 % brightness" can be the darkest setting: a driver overflow near the maximum (tutorial).
• Grafana downloads plugins on first start unless told not to (tutorial).

Found this useful? Tell us what to measure next.