Agentic AI networks module 3: agents, orchestration, routing, RAG and memory
Fifteen sessions from one model with tools to several agents that share work. You write an agent loop yourself, then look at what real harnesses add, learn the patterns that keep multi-step work predictable, route each task to the cheapest model able to do it, give agents a searchable memory with embeddings and retrieval, and connect agents that run on different machines. The module ends with a two-agent network, documented with a diagram.
Addresses in the examples are local or reserved for documentation (192.0.2.10, 198.51.100.20): replace them with your own machines, and keep your real ones out of anything you publish.
Day 30 · 02/12 — The agent loop
Goal: define an agent precisely, and tell it apart from a workflow.
Theory 1/2: reason, act, observe
An agent is a loop in which a model decides the next action, a program carries it out, and the result goes back to the model, until a stop condition. Yao and colleagues named the pattern ReAct in 2022 (reasoning and acting): interleaving reasoning traces with actions such as searches. The loop you wrote on day 21 is already one. What makes it reliable is outside the model: a step cap, a time limit, a budget, allowed tools, and a clear "done" signal.
Which element guarantees that an agent stops?
- The model deciding it is done
- A good system prompt
- A step, time or budget cap enforced by the program
Answer
The model may never decide; the program's cap always works.
Theory 2/2: workflows versus agents
Anthropic's guide "Building effective agents" (December 2024) distinguishes workflows, where models and tools follow predefined code paths, from agents, where the model directs its own process. Its advice: start with the simplest solution, often a single call with good context, and add autonomy only when needed, because it costs latency and money and is harder to test.
"Every night, summarise the new log lines and post the summary" is best built as…
- a workflow: fixed steps, one model call
- an autonomous agent with all tools
Answer
The steps are known in advance: no need for the model to choose them.
model(step) returns 'tool' or 'done'. Write runAgent(model, maxSteps): call it with 0, 1, 2…; stop on 'done' or when maxSteps calls were made. Return {steps, stopped} with stopped equal to 'done' or 'cap'.
function fakeModel(script) { return step => script[step] || 'tool'; }
function runAgent(model, maxSteps) {
}
Answer
function fakeModel(script) { return step => script[step] || 'tool'; }
function runAgent(model, maxSteps) {
for (let step = 0; step < maxSteps; step++) {
if (model(step) === 'done') return { steps: step + 1, stopped: 'done' };
}
return { steps: maxSteps, stopped: 'cap' };
}Practice (30 min)
- Read "Building effective agents" fully; list its patterns in your notebook.
- For five tasks of your own (e.g. triage mail, answer from notes, fix a failing test), decide workflow or agent and justify in one line.
- Add a time limit and a token budget to your day-21 loop, next to the step cap.
With your AI: "Here are five tasks I want to automate: [paste]. For each, ask me two questions about how predictable the steps are, then tell me whether you would build a workflow or an agent, and what cap (steps, time, tokens) you would set. I will defend or change my own choice."
Recap (10 min)
Three lines: the loop, the stop guarantees, workflow versus agent. Redo runAgent without notes.
Day 31 · 03/12 — Your own minimal agent
Goal: write a small agent that answers questions about your notes with your MCP tools, and log every step.
Theory 1/2: the parts
A minimal agent has: a system prompt (role, rules, when to stop), a tool list, the loop with caps, a trace (each message and tool result, written as JSON lines) and a final answer format. The trace is what you will read when it goes wrong, and what you will test on (module 4).
Why write the trace as JSON lines?
- It is prettier
- A program can replay, count and test it
Answer
Machine-readable traces make evaluation possible.
Theory 2/2: an answer that cites
Ask the agent to name the note each statement comes from. A citation lets code check that the note exists and you check that it says so. "I did not find it" must be an allowed answer, written in the prompt; otherwise the model fills the gap.
The agent cannot find the answer in the notes. The desired behaviour is…
- to say it did not find it
- to answer from general knowledge without saying so
Answer
Say so. Allow it explicitly in the system prompt.
Practice (30 min)
- Write
agent_notes.py: system prompt inprompts/notes-agent.txt, tools fromainet/tools.py(or through your MCP server with the SDK client), caps of 6 steps and 60 seconds, a trace file per run inruns/. - The final answer is JSON:
{"answer": ..., "sources": [note names]}, validated in code; the program checks each source exists. - Run it on five questions, including one with no answer in the notes.
With your AI: "Here is the trace of one run of my notes agent, as JSON lines: [paste]. Walk through it step by step and tell me where it wasted a step, whether each cited source really contains the claim (quote the tool result), and one change to the system prompt that would have saved a step."
Let an agent do it: now that your agent works, a coding agent can add a replay tool: "Write runs/replay.py that prints a trace file as a readable conversation, with step numbers, tool names and durations, and a test on a sample trace." Check it on a real trace you have read yourself.
Recap (10 min)
Three lines: the parts of your agent, why traces, why citations. Redraw the agent's structure from memory.
Day 32 · 04/12 — Harnesses: what they add
Goal: know what a harness does beyond your loop, and choose one per job.
Theory 1/2: the harness around the model
A harness is the program that turns a model into a working agent: it reads files, runs commands, asks permission, manages the context (summarising, trimming), handles errors, and decides when to stop. Your loop is a tiny harness. The site's guide what an agent harness is, and how to choose yours compares six in daily use; others describe Claude Code, Kimi Code and OpenFox, built for local models.
Which of these is a harness's job, not the model's?
- Choosing a word
- Writing a plan
- Asking you before running a command that deletes files
Answer
Permissions are enforced by the harness.
Theory 2/2: choosing
Criteria: which models it supports (local, hosted), permission model (allow-lists, sandbox, confirmations), MCP support, how it handles long contexts, cost, licence, whether you can run it headless (scripts, servers). A harness built for frontier models may struggle with a small local one, and the reverse.
For a nightly job on a server without screen, the key criterion is…
- a nice interface
- a non-interactive (headless) mode with fixed permissions
Answer
No one is there to click "allow".
Practice (30 min)
- Read the harness guide.
- Install one harness that can use your local model (OpenFox, or another with an OpenAI-compatible setting) and connect your MCP server.
- Give it the same five questions as yesterday's agent; compare steps, time and answers in a table.
With your AI: "I want to choose an agent harness for [my use: coding on a laptop / nightly jobs on a server]. Here are my constraints: [models, privacy, budget]. Make a table of three harnesses against these criteria: supported models, permission model, MCP support, headless mode, licence, with the documentation link for each cell. Write 'unknown' rather than guess."
Let an agent do it: you just compared runs by hand; a harness with your MCP server can now answer the notes questions for you every day. Give it: "Answer from the notes tools only, cite the note names, say 'not found' otherwise." Keep checking one cited note per answer yourself.
Recap (10 min)
Three lines: what a harness adds, your criteria, your comparison result. Redo the second exercise without notes.
Day 33 · 05/12 — Context engineering
Goal: decide what goes into the context at each turn, and keep long runs within the window.
Theory 1/2: the context is a budget
Each turn resends instructions, tool descriptions, history and results. Long runs fill the window and slow down (prefill), and models use information in the middle of a very long context less reliably (Liu et al., 2023, "Lost in the Middle"). Levers: fewer tools exposed per task, short tool outputs, summarising old turns, moving long material out (a file the agent can read on demand), and a stable prefix for cache reuse. The site measured where an agent's tokens went in the context floor.
An agent has 40 tools but each task uses three. Best lever?
- A larger context
- Expose only the tools the task needs
Answer
Every description is paid at every turn and can confuse the choice.
Theory 2/2: summarising without losing facts
When the history is too long, replace old turns by a summary written by a model, but keep exact facts (paths, numbers, decisions) in a structured note the program maintains, not in prose. A summary can drop or alter a figure.
Each message has a token count t. Write trim(messages, budget): keep the most recent messages whose total fits in the budget, in their original order.
function trim(messages, budget) {
}
Answer
function trim(messages, budget) {
const kept = [];
let total = 0;
for (let i = messages.length - 1; i >= 0; i--) {
if (total + messages[i].t > budget) break;
total += messages[i].t;
kept.unshift(messages[i]);
}
return kept;
}Practice (30 min)
- Measure your notes agent: tokens per turn (from
usage), split into system prompt, tools, history. - Reduce: shorter tool outputs, fewer tools, a shorter system prompt. Rerun the five questions and compare tokens and answers.
- Add a history budget with your
trimlogic (keeping the system prompt and the first user message).
With your AI: "Here is the token breakdown of one agent turn: system [n], tools [n], history [n], tool results [n]. Suggest three changes ranked by tokens saved per turn, estimate the saving for each with the arithmetic, and name the risk each one carries."
Let an agent do it: after measuring by hand, ask a coding agent: "Add per-turn token accounting to agent_notes.py (system, tools, history, results) written into the trace, and a script that prints the average over runs/." Check one turn against the usage figures yourself.
Recap (10 min)
Three lines: what costs tokens, three levers, why facts stay structured. Redo trim without notes.
Day 34 · 06/12 — Workflow patterns
Goal: recognise the five standard patterns and pick one for a task.
Theory 1/2: five patterns
The same guide names five workflow patterns:
- Prompt chaining: fixed steps, each call feeds the next, with checks between them.
- Routing: classify the input, then send it to a specialised prompt or model.
- Parallelisation: split a task into independent parts (sectioning), or ask several times and vote.
- Orchestrator-workers: a model plans subtasks at run time and delegates them.
- Evaluator-optimiser: one call produces, another evaluates and asks for improvements, in a loop.
"Translate, then check the translation against a glossary, then fix" is…
- routing
- prompt chaining (with a check step)
- voting
Answer
Fixed sequential steps: chaining.
"Ask three times and keep the majority answer" is…
- orchestrator-workers
- chaining
- parallelisation by voting
Answer
Voting trades cost for consistency.
Theory 2/2: put checks between steps
The value of a workflow is that code can check each step: a schema, a count, a test. A chain without checks only multiplies errors.
Write vote(answers): return the most frequent answer (the first one reaching the top count on ties).
function vote(answers) {
}
Answer
function vote(answers) {
const counts = {};
let best = answers[0];
for (const a of answers) {
counts[a] = (counts[a] || 0) + 1;
if (counts[a] > counts[best]) best = a;
}
return best;
}Practice (30 min)
- Build a chain in
chain_digest.py: (1) extract action items from a note as JSON, validated; (2) classify each by urgency with your triage prompt; (3) write a short digest. Code checks between steps. - Add voting (three runs) to step 2 and measure whether the triage score improves on your bench.
- Note cost (tokens) against gain.
With your AI: "Here is a task I want to automate: [describe]. Propose which of the five patterns (chaining, routing, parallelisation, orchestrator-workers, evaluator-optimiser) fits, draw the steps as a numbered list, and for each step name the check that code can run on its output."
Recap (10 min)
Three lines: the five patterns, why checks between steps. Redo vote without notes.
Day 35 · 07/12 — Orchestrator and workers
Goal: split work between a planning agent and worker agents, with a contract for each handoff.
Theory 1/2: who does what
An orchestrator breaks a request into subtasks and hands each to a worker with a narrower role, fewer tools, maybe a smaller model. The handoff is the weak point: the worker knows only what it is given. Each handoff should state the goal, the inputs, the allowed tools, the expected output (schema) and the done criterion. Multi-agent systems cost more tokens and add failure points: use them when subtasks are truly separable or need different permissions.
A good reason to split into two agents is…
- one part needs write access and the other must stay read-only
- it looks more advanced
Answer
Different permissions are a strong reason; fashion is not.
Theory 2/2: the code decides
Let models propose; let code decide what has one right answer (did the test pass, is the JSON valid, is the worker allowed this tool). The site's articles the deterministic doctrine and BMAD with small models show this in production: stories with a schema, scenarios run by tests, one story alive at a time.
The worker says "tests pass". The orchestrator should…
- believe it
- run the tests itself, by code
Answer
A claim is not a result. Check by code.
Practice (30 min)
- Write
orchestrator.py: the orchestrator receives a question, decides by a JSON plan which worker to call (the notes worker from day 31, or a "calculator" worker with only an arithmetic tool), and composes the answer. - Define the handoff as a JSON Schema:
goal,inputs,allowed_tools,output_schema; validate each plan in code. - Log both levels in the trace; run five mixed questions.
With your AI: "Here is my handoff schema between an orchestrator and its workers: [paste], and one plan it produced that went wrong: [paste]. Tell me which field was missing or vague and caused the failure, and propose a stricter schema rule that code could check."
Let an agent do it: having wired one orchestrator by hand, ask a coding agent: "Add a third worker, summarise_note, to orchestrator.py, with its own allowed tools, a handoff validated against the existing schema, and a test where the orchestrator plans it." Check the worker cannot use tools outside its list by trying it.
Recap (10 min)
Three lines: orchestrator, worker, handoff contract. Redo the second exercise without notes.
Day 36 · 08/12 — Routing between models
Goal: send each request to the cheapest model able to handle it, local first, hosted when it is worth it.
Theory 1/2: a gateway
A gateway gives every client one OpenAI-compatible address and maps model names to real backends (local runtimes, hosted APIs), with retries, fallbacks and logs. LiteLLM's proxy is a common open-source one. The site's articles one gateway for all your models and free cloud models behind your gateway describe a real setup.
The main benefit of a gateway is…
- faster models
- one address and one naming for every client, with fallbacks
Answer
Clients stop knowing which backend answers; you change it in one place.
Theory 2/2: deciding the route
Routes can be decided by rules (task type, length), by a small classifier, or by escalation (try local, check, escalate on failure). Keywords are a weak router: the site measured them below the trivial "always answer default" baseline in route requests with a small classifier. Hybrid escalation shows a queue that tries the cheapest agent first, with daily caps.
Write route(task): 'large' if the task is code or longer than 8,000 tokens; else 'small' for classify; else 'default'.
function route(task) {
}
Answer
function route(task) {
if (task.kind === 'code' || task.tokens > 8000) return 'large';
if (task.kind === 'classify') return 'small';
return 'default';
}Practice (30 min)
- Install LiteLLM's proxy (see its docs) with a
config.yamldeclaring two names,smallandlarge, pointing at your llama.cpp and Ollama models (or two sizes); keys, if any, come from environment variables. - Point
ainet/llm.pyat the gateway; your orchestrator usessmallfor planning and triage,largefor the final answer. - Measure total time and tokens on your five questions against "everything on large".
With your AI: "Here is my LiteLLM config.yaml with secrets removed: [paste]. Check it against the LiteLLM proxy documentation: model names, backends, fallbacks, where keys come from. List anything that would fail to start or leak a key, quoting the line, and link the documentation page for each point."
Let an agent do it: once your two routes work by hand, an agent can add a fallback: "Add to config.yaml a fallback from large to default when large fails, restart the proxy, and prove it by stopping the large backend and running one request; report the log lines." Check the log yourself.
Recap (10 min)
Three lines: what a gateway does, three ways to route, your measured gain. Redo route without notes.
Day 37 · 09/12 — Small judges for small decisions
Goal: answer typed yes/no and choice questions with a tiny model, and use its confidence correctly.
Theory 1/2: the right size for the decision
Many agent decisions are small: is this mail spam, which of three queues, is this answer on topic. A sub-1B model on a processor can answer them in a fraction of a second, if code asks a typed question and reads a probability rather than prose. The site describes one in a tiny local judge and a two-pass version in Tev1 first, Nimble when in doubt.
For "is this message about billing? yes/no", the best output is…
- a paragraph explaining the answer
- a single label with a probability
Answer
Code can threshold a probability; it cannot reliably parse a paragraph.
Theory 2/2: confidence is not correctness
A probability measures how concentrated the model's answer is, not whether it is right. Calibrate on your own labelled cases: pick a threshold, and send what falls below it to a bigger model or to a human. Two options with nearly equal probabilities compete: escalate, do not pick in silence.
Write decide(probs, threshold): return the label with the highest probability if it reaches the threshold, else 'escalate'.
function decide(probs, threshold) {
}
Answer
function decide(probs, threshold) {
let best = null;
for (const k in probs) if (best === null || probs[k] > probs[best]) best = k;
return probs[best] >= threshold ? best : 'escalate';
}Practice (30 min)
- Serve a very small model (for example
qwen3.5:0.8bin Ollama) on the processor. - Ask it your triage question constrained to one label; get probabilities with llama-server's
n_probsoption on/completion(see the server README) or by sampling ten times and counting. - On your 10 labelled cases, find the threshold above which it is always right; note how many cases fall below it.
With your AI: "Here are my judge's results on twenty labelled cases: label, probability, correct or not: [paste]. Help me choose a threshold: for thresholds 0.6, 0.7, 0.8 and 0.9, compute how many cases are accepted and how many of those are wrong, in a table. Do not choose for me; explain the trade-off."
Let an agent do it: now that you have calibrated a judge by hand, an agent can wire it in: "In orchestrator.py, call the small judge first for triage, accept its label above the threshold in config, and escalate to the 'large' route below it; log which path was taken; add a test for each path." Check the threshold comes from configuration, not the code.
Recap (10 min)
Three lines: which decisions are small, why typed answers, what confidence means. Redo decide without notes.
Day 38 · 10/12 — Embeddings
Goal: turn texts into vectors and find the nearest ones by cosine similarity.
Theory 1/2: meaning as a vector
An embedding model maps a text to a vector of a few hundred or thousand numbers, such that texts with similar meaning get close vectors. Sentence-BERT (Reimers and Gurevych, 2019) made this practical for sentence search. Locally, Ollama serves embedding models such as embeddinggemma on /api/embed; llama-server does with --embedding and /v1/embeddings. The MTEB leaderboard compares models, but your own retrieval test decides.
Can you compare vectors produced by two different embedding models?
- Yes
- No: each model has its own space; index and query with the same model
Answer
Change the model, re-embed everything.
Theory 2/2: cosine similarity
Closeness is usually measured by the cosine of the angle between two vectors: dot product divided by the product of their lengths, from −1 to 1.
Write cosine(a, b) for two arrays of the same length.
function cosine(a, b) {
}
Answer
function cosine(a, b) {
let dot = 0, na = 0, nb = 0;
for (let i = 0; i < a.length; i++) {
dot += a[i] * b[i]; na += a[i] * a[i]; nb += b[i] * b[i];
}
return dot / (Math.sqrt(na) * Math.sqrt(nb));
}Practice (30 min)
ollama pull embeddinggemma, then:curl -s http://localhost:11434/api/embed -d '{"model": "embeddinggemma", "input": ["the GPU fan is loud", "the graphics card is noisy", "buy bread"]}'- In Python, compute the cosine between the three vectors; check that the first two are closest.
- Embed every note of your notebook and save the vectors with the note names in
index/notes.json.
With your AI: "Explain cosine similarity with a two-dimensional example I can draw, then give me three pairs of short sentences and ask me to rank them by expected similarity before I compute it. After I report my measured cosines, discuss where intuition and the model disagreed."
Recap (10 min)
Three lines: what an embedding is, why one model per index, cosine. Redo cosine without notes.
Day 39 · 11/12 — RAG 1: retrieve, then answer
Goal: understand retrieval-augmented generation and its three design choices: chunks, retrieval, prompt.
Theory 1/2: the idea
RAG (Lewis et al., 2020) answers from documents instead of from memory: split documents into chunks, embed them, and for each question retrieve the closest chunks and put them in the prompt with an instruction to answer only from them and cite them. It gives the model fresh, private knowledge without training, and makes answers checkable.
What does RAG change in the model?
- Nothing: it changes what is in the prompt
- Its weights, by training on your documents
Answer
No training: retrieval feeds the context.
Theory 2/2: chunking
Chunks too small lose context; too large dilute the match and waste tokens. Split along structure (headings, paragraphs) with a size limit and a little overlap, and keep the source and position with each chunk for citations. The site's RAG on your own journal shows the chunk size that mattered and three ways a RAG silently serves stale answers.
Write chunks(text, size, overlap): cut the text into pieces of at most size characters, each starting size - overlap characters after the previous one; stop when the end is reached.
function chunks(text, size, overlap) {
}
Answer
function chunks(text, size, overlap) {
const out = [];
for (let start = 0; start < text.length; start += size - overlap) {
out.push(text.slice(start, start + size));
if (start + size >= text.length) break;
}
return out;
}Practice (30 min)
- Read the journal RAG article.
- Write
ainet/rag.py: split each note by headings, then by about 800 characters with 100 of overlap; keep{source, start, text}. - Embed the chunks, store them in
index/chunks.json, and writesearch(question, k=4)returning the closest chunks.
With your AI: "Here are three chunks my retriever returned for the question '[question]': [paste]. Tell me, for each chunk, whether it actually helps answer the question, and whether the chunk boundaries cut an important sentence. Suggest one change to my chunking rule."
Recap (10 min)
Three lines: the RAG steps, why chunk size matters, what each chunk keeps. Redo chunks without notes.
Day 40 · 12/12 — RAG 2: build and measure it
Goal: answer with citations from your index and measure retrieval with recall@k.
Theory 1/2: measure the retriever separately
A wrong RAG answer has two possible causes: the right chunk was not retrieved, or the model did not use it. Measure retrieval alone with a small set of questions whose source you know: recall@k is the fraction of questions for which the right source is among the top k results. Fix retrieval before blaming the model.
Write recallAtK(results, expected, k): results[i] is the ranked list of sources for question i, expected[i] its right source.
function recallAtK(results, expected, k) {
}
Answer
function recallAtK(results, expected, k) {
let hit = 0;
results.forEach((r, i) => { if (r.slice(0, k).includes(expected[i])) hit++; });
return hit / expected.length;
}Theory 2/2: storing vectors
A few thousand chunks fit in a JSON file and a loop. Beyond that, use a vector index: sqlite-vec adds vector search to SQLite; dedicated databases exist for millions. Whatever the store, re-index when documents change, and record which embedding model built the index.
Your recall@4 is 0.5. Before trying a bigger chat model, you…
- buy one
- work on chunking, the embedding model or the query
Answer
Half the time the model never sees the answer.
Practice (30 min)
- Write
bench/rag.jsonl: ten questions with the note that answers each. - Compute recall@1 and recall@4 for your index; try one change (chunk size or embedding model) and compare.
- Add a
search_index(question)tool to your MCP server and let the notes agent use it; answers must citesource.
With your AI: "Here are my ten retrieval test questions and, for each, the top four sources returned and the expected one: [paste]. Compute recall@1 and recall@4 and show the arithmetic; then group the misses by likely cause (vocabulary mismatch, chunk boundary, ambiguous question) and suggest one fix per group."
Let an agent do it: now that you have measured recall by hand, an agent can run the comparison grid: "For chunk sizes 400, 800 and 1,600 characters, rebuild the index, compute recall@1 and recall@4 on bench/rag.jsonl, and write a table to bench/rag-results.md; do not edit rag.jsonl." Check one row by rerunning it yourself.
Recap (10 min)
Three lines: two causes of a bad answer, recall@k, your result. Redo recallAtK without notes.
Day 41 · 13/12 — Memory
Goal: give agents memory that survives a session, without letting it rot.
Theory 1/2: kinds of memory
Within a session, memory is the context. Across sessions, it is whatever code writes and reads back: notes files, a database, a RAG index of past runs. MemGPT (Packer et al., 2023) framed this like an operating system: a small fast "main context" and a large external memory the agent pages in and out with tools. Simple versions work well: one fact per file, a short index loaded at start, the rest searched on demand.
Where does an agent's memory live between two sessions?
- In files or databases written by code
- In the model's weights, which learn as it works
Answer
Weights do not change while you use the model.
Theory 2/2: memory goes stale
A remembered fact can be superseded, and a model reading it will repeat it with full confidence. Give each memory a date and a source; when a fact changes, update the note and keep the history below; expire what can expire. The site's knowledge packs add a test and an expiry date to each claim; an agent's state as a portable file keeps a whole agent in plain files.
A memory note says "the server runs version 2". It is now version 3. Best fix?
- Add a second note "version 3" next to it
- Update the note with the current state at the top, dated, and keep the old state as dated history
Answer
Two contradictory notes make the agent pick one at random.
Practice (30 min)
- Create
memory/with one Markdown file per fact (front matter:date,source) andmemory/INDEX.md, one line per fact. - Add two tools to your MCP server:
remember(name, text, source)(writes or updates one file, dated) andrecall(query)(searches the memory folder). - Run the agent on three sessions; check what it wrote; correct one stale fact by hand.
With your AI: "Here is my agent's memory folder index and three of its notes: [paste]. Find notes that contradict each other, notes without a source or date, and notes that state something that may have changed. For each, quote it and propose the corrected note."
Let an agent do it: after cleaning memory once by hand, an agent can audit it weekly: "List memory notes older than 30 days or without a source, and for each propose keep, update or delete, without changing any file." You decide and edit.
Recap (10 min)
Three lines: memory within and across sessions, why notes go stale, how to date them. Redo the second exercise without notes.
Day 42 · 14/12 — Prompting for orchestration: intents
Goal: write the task you hand to a worker agent so that it can succeed and you can check it.
Theory 1/2: weak and edited intent
Weak intent handed to a coding worker: "Improve the RAG, it's not great." What is wrong: no measurable goal, no files, no data, no limit, no proof. Edited intent:
Goal: raise recall@4 on bench/rag.jsonl from 0.6 to at least 0.8.
Context: chunking is in ainet/rag.py (function split_note); the index is rebuilt by scripts/index.py.
Files you may change: ainet/rag.py, tests/test_rag.py. Nothing else.
Data: here are the two missed questions and their expected sources: [paste real lines].
Constraints: keep the embedding model; no new dependency.
Deliverable: the change, a test that runs split_note on a sample note, and the new recall figures.
Done when: uv run pytest -q passes and scripts/recall.py prints recall@4 >= 0.8.
The site measured what makes such handoffs work with local coding agents in giving code work to a local agent: at most four deliverables, the real data pasted, a test that executes, and a pull request someone reads.
Which line of an intent matters most for checking the result?
- The goal written politely
- The list of context files
- "Done when" with a command and a threshold
Answer
It turns "better" into something code can verify.
Theory 2/2: fix this intent
"Look in the other repository for the JSON format and adapt the parser." The most useful fix is…
- add "carefully"
- paste a real JSON sample and say not to look for it elsewhere
Answer
Give the data, not its description: a worker may waste its whole budget searching.
Practice (30 min)
- Write three intents in
intents/for real improvements to your lab (one per file), each with goal, context files, allowed files, data, constraints, deliverable, done-when. - Hand one to a coding agent (harness from day 32) and let it work.
- Check the result only by the done-when command, then read the diff.
With your AI: "Here is an intent I wrote for a coding agent: [paste]. Check it against seven parts: goal with a figure, context files, files allowed to change, real data pasted, constraints, deliverables (at most four), done-when command. For each missing or vague part, propose the exact line to add."
Let an agent do it: this whole session is the handover itself: you wrote the intent by hand, the agent does the work. What you still do yourself: run the done-when command, read every line of the diff, and refuse anything outside the allowed files.
Recap (10 min)
Three lines: the seven parts of an intent, why real data, why a done-when command. Rewrite the weak intent from memory.
Day 43 · 15/12 — Agents across machines
Goal: connect agents and models on two machines without exposing anything to the internet.
Theory 1/2: what crosses the network
A typical home network of agents: a machine with a GPU serves models; a laptop runs the orchestrator; a small always-on box runs automation. Between them travel model API calls (HTTP), MCP over Streamable HTTP, and SSH for commands. Rules: services listen on a private interface only, never on all interfaces of a machine reachable from the internet; every HTTP service needs an API key (llama-server has --api-key); traffic between machines goes through a private, encrypted network.
You start llama-server with --host 0.0.0.0 on a machine with a public address and no key. Risk?
- None
- Anyone who finds the port can use your GPU and read your prompts
Answer
Bind to a private interface and require a key.
Theory 2/2: private networks and SSH
WireGuard is a modern VPN protocol; mesh tools built on it, such as Tailscale, connect your machines as if they were on one LAN. SSH remains the tool for commands, and agents running many SSH commands benefit from shared connections; the site's SSH for an agent workstation lists four traps, among them never passing free text as a remote command's arguments.
The model server should listen on…
- the private network interface only, with an API key
- every interface, for convenience
Answer
Least exposure, plus authentication.
Practice (30 min)
- If you have two machines: connect them through a private network (WireGuard or a mesh tool), start llama-server on one with
--hostset to its private address and--api-keyread from an environment variable, and call it from the other. With one machine: simulate with two ports on127.0.0.1and a key. - Check from the server machine with
ss -tlnpwhich address each service listens on; write the table in your notebook (use documentation addresses like192.0.2.10if you publish it). - Try a call without the key and note the error.
With your AI: "Here is the output of ss -tlnp on my model server, with addresses replaced by documentation ones: [paste]. For each listening service, tell me whether it is reachable from the local machine only, the private network or everywhere, and which ones need an API key or a firewall rule. Explain how you read each line."
Let an agent do it: once you have checked your listening ports by hand, an agent can repeat the audit on each machine: "Over SSH, run ss -tlnp read-only on these hosts, list every service listening on a non-loopback address, and flag those without authentication." Verify any flagged service yourself before changing it.
Recap (10 min)
Three lines: what travels between machines, where services listen, why a key. Redo the first exercise without notes.
Day 44 · 16/12 — Module 3 deliverable: a two-agent network
Goal: publish an orchestrator and a worker that cooperate, with routing, retrieval, traces and a diagram.
Theory 1/2: what to show
A reviewer wants to see: the architecture (who calls whom, over which protocol, with which permissions), how a request flows, how routing decides, what is measured (recall, steps, tokens, time), and the failure modes you saw. A diagram plus three numbers says more than a page of adjectives.
Which belongs on your architecture diagram?
- The brand logos of each tool
- Each process, the protocol between them, and what each may read or change
Answer
Flows and permissions are what a reviewer checks.
Theory 2/2: name the failure modes
Write down the failures you met and what now catches them: a worker inventing a source (caught by the source check), a loop that would not stop (caught by the cap), a route sending code to the small model (caught by the rule). That list is the start of module 4's test set.
Your agent once looped calling the same tool. What belongs in the README?
- The failure, its cause, and the guard that now stops it
- Nothing: it is fixed
Answer
Known failures and their guards show you understand the system.
Practice (30 min)
- Draw the diagram (Mermaid in Markdown is fine): user → orchestrator → gateway → models; orchestrator → workers → MCP server → notes, index, memory.
- Write
docs/network.md: components, protocols, permissions per component, routing rules, measurements from days 36 and 40, failure modes and guards. - Run the five reference questions end to end, save the traces under
runs/module-3/, commit, tagmodule-3.
With your AI: "Here is my architecture document for a two-agent network: [paste]. Act as a security-minded reviewer: for each component, ask what it can read, change and reach on the network, and point out any component whose permissions are not stated. Then list the three questions you would ask me in an interview about it."
Let an agent do it: now that you wrote the document yourself, an agent can check it against the code: "Read docs/network.md and the code; list every statement in the document that the code contradicts or that has no code behind it, with file and line." Fix the document or the code yourself.
Recap (10 min)
Three lines: what you built, your best measurement, your worst failure. Redo the exercise of day 30 on runAgent without notes.
Sources
- Yao et al. (2022), ReAct: Synergizing Reasoning and Acting in Language Models
- Anthropic (2024), Building effective agents
- Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Reimers and Gurevych (2019), Sentence-BERT
- Packer et al. (2023), MemGPT: Towards LLMs as Operating Systems
- Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts
- LiteLLM — proxy server
- Ollama — embeddings
- sqlite-vec
- MTEB leaderboard
- WireGuard
- Hugging Face — AI agents course
Found this useful? Tell us what to measure next.