The Token Diet

00 The chart that made me stop

I have been building a lot with Claude Code, not just agentic ai projects but personal projects, mobile apps, games, enterprise style monitoring and dashboards for all my infrastructure etc. A year of that produced 55 repositories, over 50 GB on disk, a curated wiki, and 381 memory files the agent wrote for itself along the way. It also produced a token usage I was reading the way most people read a utility statement: glance, wince, move on.

Then I built a small local dashboard that parses Claude Code's own transcript files and charts where the tokens go, along with gemini and some plugin monitoring. One number stopped me. On August 16, raw input tokens were 0.019 percent of everything the model saw that day. Nearly forty thousand input tokens against 212 million cache reads. The prompt I type is a rounding error. The cost lives in what the session forces the model to read.

And what was it reading? My own work. Again. Every session started with the agent grepping through repositories it had already explored hundreds of times, re-deriving architecture it had itself written, and re-reading files wholesale to answer questions it had already answered in April. I was paying, per turn, for the model's amnesia about a corpus I had spent a year helping it create.

01 The week the price tripled

Through July the pattern was tolerable because I mostly lived inside one repository at a time. My dashboard says those days averaged around six thousand paid tokens per turn. Fine. Cache does its job when you stay in one place.

The first week of August broke that. I was doing portfolio-wide work: tracing one identity pattern across a dozen repos, comparing decisions I had made months apart, pulling threads that crossed project boundaries. The per-turn price responded immediately. August 2 through 5 averaged 15,289 paid tokens per turn, with one day peaking above 24,000. Same model, only me, triple the price, because every cross-repo question turned into a fresh archaeology dig through raw files.

That was the moment the obvious thing finally became obvious. I did not have a context problem. I had a retrieval problem. The knowledge existed, in code, in docs, in the agent's own memory files. It was just stored in the most expensive possible format for a language model to consult: loose, unindexed text that has to be re-read in full to be useful.

So on August 8 I spent one deliberately heavy day fixing that. It is the tallest bar on the chart below, 30.9 million paid tokens, and it was the day Claude and I indexed the year.

Everything to the right of the purple bar is portfolio-wide work, the same kind that cost 15,000 tokens a turn in early August. It now runs at 6,677 per turn averaged across the eight days that followed, while turn volume went up five-fold. Cross-repo questions started costing what single-repo questions used to cost.

02 Four layers, four kinds of waste

No single tool did this. Four layers went in, and each one kills a different way that tokens were being wasted. They barely overlap.

RTK: stop paying retail for shell output

RTK is a Rust CLI proxy that sits between Claude Code and the shell. A hook rewrites git status to rtk git status, and RTK returns a compressed, structured version of the output instead of the raw dump. The agent still gets everything it needs to act. It just stops receiving eighty lines of decoration around four lines of signal.

My totals since installing it: 10,620 commands proxied, 5.6 million tokens saved, 76.6 percent average compression. The single biggest line item is file reads at 4.3 million tokens saved. Nobody thinks of cat as a cost center. It is the whole grocery bill.

The code graph: structure questions without file dumps

code-review-graph parses the codebase with Tree-sitter into a persistent graph, kept current by hooks on every file change. This repo's graph is 3,213 nodes and 29,425 edges across 642 files. When the agent needs to know who calls a function, what a change would break, or whether something has test coverage, it queries the graph and gets a structural answer in a few hundred tokens. The old way was a Grep cascade followed by reading five files end to end, thousands of tokens to reconstruct a fact the graph simply knows. My project instructions now open with a rule: graph first, files only when the graph cannot answer.

Code structure is only half the graph story. A second tool, graphify, builds a knowledge graph over the whole portfolio, not just the code: concepts, documents, and components, linked across repo boundaries. This is what it looks like. Every dot is a node, every color is a detected community, and the bright hubs are the pieces the rest of the year leans on.

The stat I like most about this map is what it cost to draw: nothing. Its report shows 99 percent of the graph was extracted structurally, by parsing, with zero model tokens spent on inference. The most token-expensive thing I own to read was free to index. That asymmetry, expensive to re-read but nearly free to map, is the entire reason this stack works.

The knowledge base: a year of answers, indexed

This is the layer that changed cross-repo work. A local FastAPI service holds a Qdrant vector index over three roots: the curated wiki, every markdown file across all 55 repositories, and the agent's own memory files. Embeddings come from nomic-embed-text running under Ollama on the same machine. Nothing leaves the box. As of this writing the index holds 2,904 files in 35,279 chunks, and it reindexes itself daily, incrementally, touching only what changed.

Claude reaches it through an MCP tool. Ask a question that touches history, "how did I wire the token exchange policy in the banking demo," and instead of an expedition through repositories, the agent runs one kb_search call and gets ranked excerpts with citations back to the source files. It reads eight results, not eight repos, then opens only the one file that matters.

Because the index build was itself a project, I instrumented it. The numbers from the validation run are the part of this story that surprised me most, so here they are, measured on the laptop that ran it:

Read those incremental numbers again, because they carry the whole economic argument. A full rebuild costs twelve and a half minutes, once. After that, keeping a year of work searchable costs about two seconds per change. The daily reindex usually finds a handful of edited files, re-embeds only those, and is done before the coffee order. Retrieval is the same shape: every search sifts all 34,000 chunks, fuses the semantic and keyword rankings, and hands back eight excerpts in about a second. The expensive thing happens once. The cheap thing happens forever.

The instrumented run also caught things I did not ask it to find. It flagged 1,012 duplicate files sitting in 157 groups across the corpus, a year of copy-paste debt I had never noticed, and it confirmed 70 separate CLAUDE.md instruction files are part of what gets indexed, which means the agent's own operating rules are retrievable the same way its work is.

Memory: lessons that stop being re-learned

The quietest layer. Claude Code writes memory files as it works: what broke, what the fix was, which approach I rejected and why. There are 381 of them now, and they are indexed by the knowledge base like everything else. A lesson learned in March costs a few hundred tokens to recall in August. Without the index, recalling it meant either re-reading the history or, worse, re-living it.

03 What the numbers did

Two of these deserve a closer look, because they are the ones I would have gotten wrong by intuition.

First, the 56 percent. The honest comparison is not July versus August. July was cheap because I stayed inside single repos, and any measurement that ignores that is flattering itself. The fair comparison is early-August portfolio work without the index, 15,289 paid tokens per turn, against the same kind of work after it, 6,677. Same breadth of questions. Less than half the price. And I did five times as many turns in the after window, so the total did not just hold, it held under load.

Second, the 0.019 percent. My daily cache reads run in the hundreds of millions of tokens, and on the dashboard that green chart towers over everything else. That used to look alarming. It is the opposite. Cache reads cost roughly a tenth of fresh input, so a tall green bar means the expensive context was written once and re-used all day. The number to watch is cache writes, the orange bars, because that is what you pay full price to create. The whole strategy reduces to one sentence: write less context, read it more often.

04 Why local infrastructure is the right shape for this

Everything in this stack runs on my own hardware. The embeddings model, the vector store, the graph, the compression proxy, the dashboard that measured all of it. That was not a privacy reflex. It follows from what this data is.

The corpus is my working life: every repo, every decision record, every memory file the agent wrote about my mistakes. It changes hundreds of times a day, which means indexing has to be cheap enough to run constantly and close enough to watch the filesystem. And the queries against it happen mid-turn, inside an agent loop, where a 40 millisecond local call and a 400 millisecond remote call are the difference between a tool the model reaches for by habit and a tool it avoids.

There is also a quieter reason. Retrieval only earns trust if you can see it. My dashboard reads the transcripts, the search service reports exactly which files back every answer, and when a result looks stale I can watch the reindex fix it. A local system lets me audit the loop end to end. I would not have found the 0.019 percent number, or the per-turn regression that started all this, inside someone else's billing page.

05 If you start today

A year of accumulated work made my version of this dramatic, but none of the layers require a year of material. Order matters, though. This is the sequence I would run with hindsight:

• Instrument first. You cannot cut what you cannot see. Point a parser at your agent's transcripts and get per-turn paid tokens charted before touching anything else. My every claim in this post came from that one afternoon of plumbing.
• Compress the shell. RTK or anything like it. It is a pure win with no behavior change, and shell output is almost certainly a bigger line item than you think. Mine was 5.6 million tokens.
• Write memory down. Start the discipline of durable lesson files now, because every layer you add later gets more useful as the corpus grows. The corpus is the asset. The tools are just ways of reading it cheaply.
• Graph the code. Structure questions are the most common expensive queries an agent makes. A code graph converts them from file reads into lookups.
• Index everything last. The vector index is the highest-effort piece and the one that improves most with corpus size. By the time you build it, the other layers have been feeding it material.

The framing I keep coming back to: a year ago I thought the output of all this work was the demos and the repos. The output was actually a corpus that knows how I build things. The moment I started treating that corpus as infrastructure, with an index, a graph, and a meter, the same agent got cheaper, faster, and noticeably less repetitive. The model did not get smarter in August. Its library got a card catalog.