The token tax of grep-and-read
AI coding agents burn their token budget re-reading code they've seen. A resident code graph replaced grep-and-read and cut usage ~82% in our benchmark.
An AI coding agent spends most of its token budget re-reading code it has already seen. Replacing that grep-and-read loop with a queryable, resident graph of the repository cut token use by about 82% and warm lookups by 15.8× in our benchmark — without changing the model.
The problem
Watch how an agent navigates a codebase today: it greps for a symbol, opens a file, reads a few hundred lines to find the definition, greps for a caller, opens another file, reads again. Every one of those reads is raw text pushed back through the model. On a real repository the loop balloons: the agent burns calls and context window on retrieval, not reasoning, and the useful signal — "where is this defined, what calls it, what does it depend on" — is buried in the surrounding lines it had to read to find it.
The obvious fixes each have a catch. Bigger context windows just raise the ceiling on a linear cost. Vector search over code chunks returns similar-looking text, not the structural answer ("the callers of this function"), and it re-embeds on every change. Neither removes the re-reading; they make it cheaper per token or fuzzier.
What we did
We built CGraph, an open-source engine that parses a repository into a queryable graph once — symbols, definitions, references, and dependencies as nodes and edges — and keeps it resident. Instead of grep-and-read, the agent asks the graph a structural question and gets back exactly the nodes it needs.
The decision that mattered was graph over embeddings. Code has an exact structure that a graph can answer precisely and incrementally: "callers of X" is a traversal, not a similarity score, and a file change updates a few edges rather than re-embedding a corpus. We gave up semantic fuzziness — the graph won't find "code that feels related" — in exchange for exact answers and near-zero query cost. For an agent that already has the model's reasoning, exactness is the scarce resource.
The result
Across four realistic navigation tasks, the graph path used 3,966 tokens across 4 calls where a grep-and-read agent spent 22,373 tokens across 10 calls — about 82% fewer tokens. A warm structural query returned in 10.6 ms versus 167 ms (15.8× faster), and building the graph took 0.42 s where the comparison tool took 83 s. The savings compound on the questions that hurt most: transitive impact analysis — "what breaks if I change this?" — collapses from a tedious manual trace across many files into a single query. The full methodology and numbers are in the CGraph benchmark on NxtSoft Labs.
Takeaway
The cheapest token is the one you never send. For agents working in real codebases, the win isn't a bigger context window — it's giving them a structural index so they stop paying to re-read. If you're building AI systems that work in production, budget for retrieval architecture, not just the model.