Homelab 15: Home Assistant with a local voice, in French

Level AdvancedEstimated time 2 hRead first Docker with access to your GPU, One gateway for all your models, and three ways a local model freezes your agentsA word you don't know? Glossary

Home Assistant's voice assistant, Assist, is built from interchangeable parts: speech to text, an agent that understands, text to speech. Each part can run at home. Ours answers in French, on our own graphics card, and falls back to the processor when a game or a model takes the card.

How these figures were obtained. Measured on our own machines (see the protocol), usually a single run, with speeds as reported by the server itself; we do not publish raw logs. Treat them as orders of magnitude on this hardware, not as guarantees on yours. Figures from outside are labelled with their source.

Home Assistant hears, thinks and answers without the cloud. Speech goes through one relay: the GPU pair answers when the card is free, the CPU pair when a game or a model holds it. Slower, never mute.

A bit of history

Home Assistant was started in 2013 by Paulus Schoutsen to control a home without depending on each brand's cloud. In 2023, its makers declared "the year of the voice": the Wyoming protocol to plug speech-to-text and text-to-speech engines, and Piper, a fast local voice. Speech can now go from microphone to light bulb without leaving the house.

1. The speech parts speak Wyoming

Home Assistant talks to speech engines over the Wyoming protocol. The Rhasspy project publishes ready containers: wyoming-whisper for recognition (port 10300) and wyoming-piper for synthesis (port 10200). Add them in Home Assistant with the Wyoming Protocol integration, then build an Assist pipeline that uses them.

wyoming-whisper:  --model small-int8 --language fr --beam-size 1
wyoming-piper:    --voice fr_FR-siwis-medium

That pair runs on the processor alone. It is the floor: always there.

2. A better pair on the graphics card

On a GPU, Whisper large-v3-turbo recognises far better, and Kyutai's TTS speaks far more naturally. Measured on our card: both together use 3.9 GB of VRAM; on 72 French test clips, turbo made 8.3 % word errors in 0.08 s, while large made 8.5 % in 0.32 s. Turbo is as good and four times faster: we kept it.

3. One address, two backends

Home Assistant knows one address per service. A small relay listens there and connects to the GPU pair when it answers, otherwise to the CPU pair. When a game takes the card (see Free your graphics cards for AI), the GPU pair is stopped and the house keeps talking, just a little slower.

4. A local model to understand

For the agent, point Home Assistant at your local model server through a conversation integration that speaks the OpenAI API (several exist in HACS: choose a maintained one). Give the model the entities it may control, and nothing more.

Traps we paid for

A Docker published port accepts connections before the program behind it is ready. Our relay saw the GPU backend "up" during its 30-second warm-up and sent requests into the void. The fix: run the backend on the host network, on the loopback address, so the port refuses until the server really listens, and the relay falls back.

After a reboot, a GPU container created earlier may no longer reach the card (a device number changes at each boot). The relay then quietly serves the CPU pair: degraded, not broken. Recreate the container, or regenerate the GPU description at boot.

The model compiles parts of itself at start and needs a C compiler in the image. Without it, the GPU backend never starts.

Sources

Found this useful? Tell us what to measure next.