Homelab 9: your own search engine, for you and your agents

SearXNG is an open-source metasearch engine: you ask it, it asks Google, Bing, DuckDuckGo, Wikipedia and dozens of others, and merges the answers. The engines see your server, not you: no account, no history tied to your name. And for an agent network it does one more thing: it answers in JSON, so your local agents can search the web without any API key.

A bit of history

searx, born in 2014, is a metasearch engine: it sends your query to many search engines at once and merges the answers, without profiling you. SearXNG, a maintained fork started in 2021, carried the idea on. Hosted at home, it also becomes the search tool your local agents use, with no API key and no quota.

1. Run it

The SearXNG project maintains a ready-made Docker setup (the searxng-docker repository). Follow it, publish the service through your HTTPS proxy (Homelab 2) or keep it on your private network (Homelab 6), and set your browser's default search engine to it.

2. Two settings that matter

In settings.yml, change server.secret_key to a long random value: SearXNG refuses to start with the default one, on purpose.

Then, for your agents, allow the JSON format next to HTML:

search:
  formats:
    - html
    - json

An agent can now call https://search.yourdomain.tld/search?q=…&format=json and read the results.

Traps

A public SearXNG with JSON enabled is a free search API for every bot on the internet, and the engines will soon block your server for the traffic. Keep the JSON instance on your private network, or turn on SearXNG's limiter.

Some engines block or throttle shared instances from time to time. When one engine's results disappear, check its status in the preferences page before suspecting your setup.

Sources

• SearXNG, official repository
• SearXNG documentation

Found this useful? Tell us what to measure next.