Knowledge packs: facts with a source, a test and an expiry date

Ask a model how to install a tool and it answers with the version it saw during training, maybe two years old, with full confidence. A search on the web helps once. A knowledge pack keeps the answer checked: what was verified, where it came from, how to test it on your machine, and until when it can be trusted.

What you get. Agents that stop reconstructing procedures from memory: they read a pack whose claims are sourced, whose steps were run on the target machine, and which reminds you by itself when a fact has expired.

1. Five rules

1. A claim without a dated source does not enter a pack.
2. A procedure without an executable assertion is not a procedure.
3. A pack stays a draft until a run on the real machine has produced evidence.
4. Claims expire: each carries a lifetime, and a scheduled job opens an issue when it is passed.
5. Nothing merges on red.

2. What a pack looks like

packs/install-forgejo-runner/
├── kp.md          # front matter, sourced claims, the procedure as a template
├── assert.sh      # assertions per phase: PASS/FAIL, exit code non-zero on FAIL
└── evidence/      # what the validation run printed: versions, assertion logs

3. Its life

need → research by a capable model with web access
     → pull request, status: draft            (CI: lint + shellcheck)
     → field run by a human                   (assert.sh, phase by phase)
     → evidence committed, "validated_on" filled
     → human review → merge = validated
     → expiry job → issue → check again → update

Three roles, never mixed: the author (a model with fresh research) writes and never merges; the runner (any local model) uses a pack and never modifies it; the human runs the field test and approves.

Traps

An assertion that only greps a file for a word proves the word is there, not that the step worked. Assert on what the system does: a version printed, a port that answers, a service that runs.

Keep evidence even when it is boring. The day the pack expires, the old output is what you compare against.

Found this useful? Tell us what to measure next.