aovestdipaperino/tokensave

tokensave – Semantic Code Intelligence for AI Coding Agents

What it is – tokensave is a Rust‑based, locally‑run service that builds a semantic knowledge graph of a codebase. AI‑powered coding assistants (Claude Code, Codex, Gemini, Cursor, etc.) can query this graph instead of grepping or reading files, which dramatically reduces the number of LLM tokens and tool calls they need to understand a project.

---

Core idea

1. Index once – tokensave init parses every source file with Tree‑sitter, stores symbols, relationships and snippets in a libSQL (SQLite‑compatible) database.
2. Query via MCP – The service exposes a set of MCP (Modular Code‑Processing) tools. An agent asks, for example, tokensave search "auth" and receives a structured answer containing the entry point, related symbols and the relevant code snippets in a single call.
3. Local‑only – All processing stays on the developer’s machine; no API keys or external services are required.

---

Highlighted features (as described in the README)

Category	Details
Smart context building	One tool call returns entry points, related symbols and code snippets, eliminating the need for multiple grep/find calls.
Semantic search	Look for concepts (e.g., “authentication”) and get all symbols that implement it (login, validateToken, AuthService).
Impact analysis	Traverse callers/callees and see the full impact radius of a symbol before editing.
80+ MCP tools	Call‑graph traversal, dead‑code detection, test‑mapping, complexity metrics, etc.
50+ language support	Rust, Go, Java, Python, TypeScript, C/C++, Swift, Svelte, Astro, WGSL/HLSL/Metal, CUDA/HIP, Markdown and many more.
12+ agent integrations	Pre‑built hooks for Claude Code, Codex CLI, Gemini CLI, Cursor, Copilot, Kiro, Grok Build, and others.
Multi‑branch indexing (optional)	Separate graph per Git branch; switching branches is instant and never forces a full re‑index.
Crash‑resilient sync	File parsing runs in short‑lived subprocesses; a segfault in a Tree‑sitter grammar only kills the worker, not the whole sync.
Atomic edit primitives	Safe, AST‑aware replace/insert operations that automatically re‑index the changed files.
Cross‑session memory	Tools to record design decisions and code‑area usage, queryable across sessions.
100 % local & privacy‑first	No data leaves the machine; works behind firewalls and on air‑gapped systems.

---

Quick start (install)

Platform	Command
macOS (Homebrew)	brew install aovestdipaperino/tap/tokensave
Windows (Scoop)	scoop bucket add tokensave https://github.com/aovestdipaperino/scoop-bucket && scoop install tokensave
Any (Cargo)	cargo install tokensave (default full feature set) cargo install tokensave --features medium (medium tier) cargo install tokensave --no-default-features (lite binary)
Pre‑built binaries	Download the appropriate archive from the latest GitHub release and add the executable to your PATH.

---

Typical workflow

1. Install the binary (see above).
2. Register with your AI agent – run tokensave install --agent <name> (e.g., tokensave install --agent claude). The command writes the necessary MCP server entry and hook configuration into the agent’s native config files.
3. Initialize the index – cd /path/to/project && tokensave init. This creates a hidden .tokensave/ folder containing the libSQL database.
4. Keep it fresh – after each commit (or manually) run tokensave sync. If you installed the optional Git hooks, this happens automatically.
5. Query from the agent – the agent now calls MCP tools like tokensave search, tokensave callers, tokensave impact, etc., and receives concise, structured results instead of raw grep output.

---

Multi‑branch support (optional)

• Enable per‑branch databases with tokensave branch add (or let the automatic post‑checkout hook do it).
• Use tools such as tokensave_branch_search and tokensave_branch_diff to query across branches without checking them out.
• Branch metadata can be cleaned with tokensave branch gc.

---

Safety & robustness

• Subprocess‑isolated extraction – parsing crashes are contained; the sync continues.
• Configurable crash handling – disable the subprocess pool with TOKENSAVE_DISABLE_SUBPROCESS=1 if desired.
• Hook opt‑out – set TOKENSAVE_DISABLE_GREP_HOOK=1 to let raw grep/find run unchanged.

---

Who might use it?

• Developers who rely heavily on LLM‑based coding assistants and want to cut down on token costs.
• Teams that need deterministic, offline code‑intelligence (e.g., regulated industries).
• Anyone working with large, multi‑language monorepos where scanning the whole tree on every AI request is prohibitive.

---

Where to learn more

• GitHub repository – https://github.com/aovestdipaperino/tokensave (issues, source, releases)
• Crates.io – https://crates.io/crates/tokensave (binary releases, version history)
• Documentation – the repo includes docs/BRANCHING-USER-GUIDE.md and inline help (tokensave --help).

---

Bottom line – tokensave provides a drop‑in, fully local indexing service that lets AI coding agents answer “what does this code do?” with a single, cheap tool call, saving tokens, reducing latency, and keeping your proprietary code private.