A whole file is often the wrong unit
A developer names a method. An agent opens a file, follows imports, and collects large pieces of nearby code. Some of that context matters; much of it competes with the thing the developer actually asked about.
TokenMuncher starts with the symbol. It scans source files, extracts structural declarations with Tree-sitter, and indexes locations and hashes in SQLite. Source bodies stay on disk.
Retrieve with a purpose
The CLI resolves references such as @PaymentService.processPayment. Java overloads and duplicate names can be ambiguous, and the tool reports that ambiguity instead of silently picking a match.
Context profiles differ by intent: explaining a method, preparing a change, or inspecting a class. Each included chunk has a reason. When the token budget is tight, omissions and truncations are reported explicitly.
What works today
The usable CLI includes scanning, incremental indexing, resolution, bounded context, and a baseline benchmark. On a 207-file mixed workspace, it indexed 906 symbols; indexing unchanged files again parsed zero files.
On the small mixed-language fixture, bounded retrieval used approximately 54% fewer candidate source tokens than name-based file dumps. That is fixture evidence, not a production-wide savings claim. Context savings only matter when the necessary source remains present.
What comes next
The current release works at the syntax level. Planned next layers include semantic relationships through Python and Java language servers, a warm background service, and an MCP integration.
The deeper question keeps me interested: how do you make an agent’s working set smaller without making its understanding smaller?