Hardware

Articles to help choose, build and optimize the machine that runs AI and services at home.

• Before you start: hardware, honestlyWhat fits on which card, with measured numbers, and what to buy first.
• Free your graphics cards for AI, and still playPut the desktop on the integrated GPU, give every byte of VRAM to the models, and let Steam borrow a card only while you play.
• Several models, two graphics cards: a grid instead of a juggling actOne container per model and per card, swaps that cost half a second per gigabyte, a voice service that gives way, and why a model must never fill its card.
• RAM, zram and swap on an AI machine: the three thieves that cost us a factor of tenBefore suspecting the GPU, measure the RAM. Three settings were eating it; fixing them took a model from 1.9 to about 20 tokens per second.
• A second graphics card through OCuLink: more VRAM without a new motherboardWhen the motherboard has one big slot, an M.2 slot can lend its four lanes to an external card. What worked, what did not, and how to read the link.
• Speculative decoding (MTP): up to twice as fast on the same cardA small head guesses the next tokens, the model checks them in one pass. The right flags, the value to measure yourself, the real gain in production, and the crash to watch for.
• A 177-billion-parameter model on two consumer cards: placement beats bandwidthA mixture-of-experts model that does not fit in VRAM can still run at home. Where its weights live matters more than the PCIe link, and two settings decide between a fast server and a frozen machine.
• Slow for no reason? It is re-reading: the KV cache trapA local model that generates fast but takes ten minutes to answer is not slow: it is reading the prompt badly. One flag made it 21 times slower, silently. How to measure it in two minutes, and the two settings to use instead.
• Long context on a 16 GB card: KVMem, and the build we runAgents read long conversations. KVMem keeps the whole history in system RAM and only the relevant blocks on the card: 48 out of 48 recall checks, flat speed past 100k tokens, 4.9 GB of VRAM saved. And what our build adds: one binary for two card generations, experts in host memory, token probabilities, a GPU expert cache that runs a 177B model from a single card.
• Which local model for which job: a bench that measures real workBenchmarks favour the fast model; real work favoured the reliable one. A bench where code judges every answer, then each model works for real on a small repository. What it showed: small and well-framed beats big and vague, and four settings matter more than the model.
• Strata: a 177-billion-parameter model on one gaming cardStrata is an open-source engine built to run Qwen3.8-Flash-Next on an ordinary PC: one 12 to 16 GB card and enough RAM for the experts. How we plugged it into our GPU grid as one more slot, and the three details that decide whether it starts.
• Your graphics card at x4 instead of x16: diagnose it, and get the x16 backA card can silently run on a quarter of its lanes after a reboot. How to read the link the firmware really gave it, why the BIOS has no 'x16' setting to pick, the one wrong fix to avoid, and what a power cycle did for us: from x4 back to x16, with nothing changed.

Found this useful? Tell us what to measure next.