n8n: an agentic clock that notices what stopped

Level AdvancedEstimated time 1 hRead first Homelab 14: automate without a Zapier subscription, with n8n and a decision cascadeA word you don't know? Glossary

Automations fail quietly. A backup that stops running, a mail sorter that crashed, an agent queue that froze: nothing breaks loudly, the job just stops. An agent network needs a clock that knows what should have happened, and says so when it did not. Ours is three small pieces.

Every job declares its cadence in a registry. Once a minute the tick compares each job's last sign of life with its period plus slack, and writes an event. A watchdog on another machine checks that the tick itself is alive.

1. A registry: what should happen, and how often

A table in PostgreSQL lists every job with its expected period, a slack, and the severity of a miss, for example:

create table clock_registry (
  job       text primary key,
  kind      text,      -- 'cadence' or 'on_demand'
  period_s  int,       -- expected interval between two signs of life
  slack_s   int,       -- tolerance before it counts as late
  severity  text,      -- 'warning' or 'critical'
  note      text
);

Adding a job to the watch is one row, not a new workflow.

2. The tick: once a minute, compare

A workflow triggered every minute reads the registry and each job's last sign of life (n8n's own workflow statistics, or the job's last event), and computes a status: ok, late, stuck. It writes one event per job into an events table, with the severity from the registry. A separate workflow turns critical events into a phone notification (Homelab 10).

Two database accounts, not one: the tick reads with a read-only account and writes with an account that can only insert events. A bug in a Code node cannot wipe a table it cannot touch.

3. The watchdog: outside n8n

If n8n itself stops, the tick stops with it, and nothing inside n8n can report it. So a tiny script on another machine asks every two minutes for the age of the last tick (a webhook that returns it), and sends an alert itself when it is too old. Who watches the watcher: something that does not depend on it.

Traps we paid for

A job that only speaks when something is wrong (a certificate check that stays silent while all is fine) must be declared on_demand, not on a cadence. Otherwise its silence looks like death and your phone rings for nothing.

"Running for a long time" is not "stuck". Workflows that do not keep their successful runs can leave rows marked running. Only call a job stuck if it also stopped succeeding within twice its period.

Read n8n's workflow statistics for signs of life, not the full execution table: several workflows keep no successful executions at all, and you would conclude they never run.

Restarting n8n made our clock raise two false critical alarms. Warn whoever reads the alerts before you restart it.

Found this useful? Tell us what to measure next.