Paritok-official/paritok-4b-v1

Paritok — Token‑saving proxy for coding agents

What it is – Paritok is an open‑source, drop‑in proxy that sits between a coding‑assistant (Claude Code, Cursor, Codex, OpenHands, etc.) and the underlying LLM API. It rewrites each request to shrink three major sources of token usage:

1. Tool‑schema filter – keeps only the tools that are actually relevant to the current user intent, replacing the rest with lightweight stubs.
2. Content compression – runs a 4 B‑parameter model (trained on 45 K real agent trajectories) that compresses file reads, tool outputs, and stale history to ~26 % of their original size, tagging them with a [REF:id] placeholder.
3. History summarisation – when the conversation exceeds the model’s context window, older turns are summarised so the session stays within the window.

All compression is non‑destructive: the agent can request the original text at any time (read_original/expand_context). The proxy requires no code changes in the agent—just point the agent’s BASE_URL (e.g., ANTHROPIC_BASE_URL or OPENAI_BASE_URL) at the Paritok server.

---

Key components

Component	What it does	How it’s delivered
Paritok gateway	HTTP middleware that rewrites requests, forwards them to the real LLM, and expands compressed references on demand.	Python package (paritok[proxy]), runnable as paritok up / paritok proxy.
4 B compression model	Learns to keep identifiers, paths, error messages, etc., while discarding surrounding noise.	Distributed on Hugging Face (paritok/paritok-4b-v1) and usable via Ollama (GGUF) or vLLM.
Embedding‑based tool filter	Uses a tiny CPU‑only embedding model (BAAI/bge-small-en-v1.5) to rank tool schemas and keep only the relevant ones.	Installed with paritok[toolselect].
Dashboard / VS Code extension	Live view of token savings per request and a UI for toggling features.	Separate repo (paritok-vscode) and /stats endpoint.

---

Reported savings

• Single‑turn: ~25 % fewer input tokens (typical Claude Code turn drops from ~96 K to ~72 K tokens).
• Multi‑turn: Savings compound—by turn 5 the reduction is 39 %, by turn 10 ~54 %, and the system can reach a **72 %** ceiling on a 200 K context budget.
• Cost impact: On Claude Sonnet ($3 / M input tokens) a 20‑turn session falls from $0.75 to $0.28.

---

Quick‑start (self‑hosted, no clone needed)

# Install the proxy (includes the gateway and CLI)
pip install "paritok[proxy]"

# Pull the model via Ollama (first run only ~2.5 GB)
paritok up          # pulls `paritok-4b-v1` and starts the proxy on port 8080

Leave that terminal running, then in a separate shell point your agent at the proxy:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080   # Claude Code / Cursor / …
# keep your normal provider API key (e.g. ANTHROPIC_API_KEY) set

The agent works unchanged; Paritok silently compresses the request before forwarding it.

---

Deployment options

Option	Hardware needed	How to run
Self‑host (default)	CPU for the tiny embedding model; GPU optional for the 4 B model (via Ollama or vLLM).	paritok up (Ollama) or vllm serve … + paritok proxy.
Hosted GPU service	None – use the SaaS endpoint at paritok.com.	Set use_gpu_server: true in paritok.yaml and provide the API key.

Both are Apache‑2.0 licensed and free to run locally.

---

Who might benefit?

• Developers using coding agents who repeatedly read large files or invoke many tools (e.g., debugging, code‑base audits).
• Teams with long, multi‑turn sessions – the token savings translate directly into lower cloud‑LLM bills and more turns before hitting context limits.
• Anyone wanting a plug‑and‑play token‑reduction layer without modifying the agent code.

---

Links from the README

• Paper – arXiv: 2608.24188
• Model – Hugging Face Hub paritok/paritok-4b-v1
• Discord community – https://discord.gg/SeBJE5Eucp
• VS Code extension – https://github.com/Paritok-official/paritok-vscode

---

Bottom line: Paritok is a genuine, open‑source tool that lets coding‑agent users keep the same LLM while cutting input‑token costs dramatically through selective tool filtering, model‑based compression, and smart summarisation.