turnstile: one model per card, and a service that never lies about what's running

Last reviewed: 2026-10-22

On a machine running several local models, everything wants the GPU's memory: the chat model, the code assistant, image generation, a game in the evening. You describe "who gets what" in files, and those files drift — they claim a 27B is loaded when a 3B is answering on the port. turnstile is our answer: a small turnstile gate that decides who gets the card, and says so honestly.

What you get. One model at a time per card, requests queued instead of trampling each other, and a turnstile status that shows what's actually running — container alive, port answering, memory consistent, model identity read from the server's log. Zero dependencies: Python 3, Docker, user systemd, no root.

1. What turnstile is not

It is not a model server. llama.cpp (or another engine) serves models in Docker containers — one container per slot, named like the slot. turnstile does exactly three things: load the requested slot (stopping whatever held the card), queue requests during a load, and verify in a loop that reality matches what it claims. That modesty is deliberate: when everything breaks, there is only one place to understand.

2. Install

Requirements: Linux with systemd and Docker, Python 3.10+, an NVIDIA card with nvidia-smi (without a card, turnstile still runs and manages CPU slots only).

git clone https://github.com/bengua/turnstile
cd turnstile
./install.sh

Everything installs for the current user: commands in ~/.local/bin, services in ~/.config/systemd/user, an example config in ~/.config/turnstile/. Nothing starts by itself — you decide when.

3. Describe your slots

One line per slot, in ~/.config/turnstile/containers.map:

name|port|card|weight|arguments

card is an index (0, 1…), all for a model spread across cards, or cpu. Start from examples/containers.map, rename your containers to match your slots, and that is all turnstile will ever know about your fleet — the rest, it measures.

4. First run

systemctl --user enable --now turnstiled
turnstile status        # what's ACTUALLY running, verified facts per slot
turnstile slots         # what could run
turnstile coder-c1      # load a slot: already_serving / served / queued

The first status often surprises: that's when you discover that what was running is not what you believed.

5. The life-changing habit: the card is asked for

A game or ComfyUI wants the card? It asks instead of taking:

turnstile free-c0       # release card 0, cleanly
turnstile restore       # afterwards: puts back exactly what was running

No more hand-killed sessions and models that never come back. Here, image renders and game sessions go through this, and the 27B takes its seat back on its own afterwards.

6. For automation

The same thing exists over HTTP on port 8130: GET /state, GET /slots, GET /counters, POST /slot. That's what our agents use to request a model instead of guessing. And if you put free-quota providers behind the gateway, the warden that greys out an exhausted model is covered in quota-warden.

Pitfalls

One container = one slot = one name. If your container is called llama-prod and your slot coder-c1, turnstile won't guess anything: rename the container, not the rule.

By default the API listens on 0.0.0.0 — handy so other lab machines can request slots, but restrict it with TURNSTILE_BIND=127.0.0.1 if the machine is exposed.

NVIDIA is tested daily; AMD and Intel are written from documentation, untested. The guardrails (30 switches/hour max, 20s of activity before any automatic switch) are tuned via environment variables with systemctl --user edit turnstiled.

Found this useful? Tell us what to measure next.