Local-first AI: retrieval and code intelligence on the dev's machine

The retrieval layer an AI coding agent depends on does not need to live in anyone's cloud. CGraph runs code intelligence entirely on the developer's machine — a resident graph daemon per repository, supervised by the operating system — so structural queries stay warm at ~10ms and the code being indexed never leaves the laptop.

The problem

Everything fast about a resident code graph depends on the daemon being up. CGraph's graphd builds the graph once, holds it in memory, watches the tree, and folds edits in as you work — that is what makes impact analysis interactive instead of rebuild-and-wait. But a plain daemon is deliberately polite: it shuts down after 5 minutes idle so per-project daemons don't linger. Come back from lunch and your first query pays the cold-start build. Nothing starts a daemon for the repo you cloned yesterday, restarts one that died, or brings any of them back after a reboot.

This matters more for agents than for humans. An agent doesn't retry politely: if its first structural call hits a cold daemon and waits out a full build, it may conclude the tool is slow and quietly stop reaching for it.

What we did

We handed the lifecycle to the platform's service manager instead of a shell script you have to remember. One command:

cgraph daemon install --search-root ~/code

registers a supervisor with macOS launchd plus one LaunchAgent per tracked repository, each running a resident graphd in never-idle mode. KeepAlive restarts a daemon that crashes; RunAtLoad brings the whole set back at login. The supervisor re-scans on an interval and reconciles: new repos gain a daemon, removed ones are cleaned up, and reconciliation is idempotent — run it twice with nothing changed and the second run touches nothing.

The decision that mattered was discovery by .mcp.json, not guessing. The supervisor tracks exactly the directories whose .mcp.json already registers a cgraph MCP server — the same file you created when wiring CGraph into your agent. We rejected heuristics ("looks like a code repo") because a background process touching every repo you own should be deterministic and auditable: a repo's discovery key, its LaunchAgent label, and the endpoint the MCP tools connect to all agree by construction. Two guarantees follow: the supervisor never mutates a repo (it only reads the tree), and uninstall removes every managed agent with no residue.

The result

With supervision in place, the cold-start penalty disappears from the agent's decision entirely: a query at 9am Monday is as warm as one mid-refactor. In our published benchmark, a warm structural query returns in 10.6 ms versus 167 ms for a tool that reloads the graph from disk — and the whole path runs on the developer's machine. No source is uploaded for indexing, no embeddings sit in a vendor's store, and the retrieval layer keeps working on a plane.

One honest caveat, stated in the tool itself: launchd-based supervision is macOS-only. On Linux the daemon is portable — you run graphd directly or supervise it with a systemd user unit; the supervisor is the launchd-specific part.

Takeaway

Local-first is not a compromise position for AI tooling — for code intelligence it is the fast path and the private path at once. If your team is weighing where an agent's retrieval layer should live, that is an architecture decision we help teams make before the first vendor contract is signed.