samuelfaj/distill

Distill – a terminal‑based coding assistant

What it is – Distill is a command‑line tool that lets you talk to a large language model (LLM) while it reads your codebase, edits files, runs shell commands, and keeps the whole interaction inside the terminal. It acts as a thin harness around any OpenAI‑compatible model (local or hosted) and also integrates with Grok, ChatGPT and OpenRouter.

Key features

• Multi‑model tiers – you can configure three roles:
  • Reasoning model – the main, heavyweight model that does the heavy‑lifting reasoning and code changes.
  • Worker model (optional) – a lighter model that can handle cheaper calls within the same conversation.
  • Utility model – a specialised model for bounded tasks such as extracting values, summarising logs, or compressing text.
• Automatic routing (Jev) – an internal decision layer decides, per request, which tier should handle the call, how much “effort” (model size/temperature) to use, and whether to fall back to a utility model. The routing logic is transparent and can be inspected via GROK_LOG_JEV=1.
• Provider‑agnostic – works with Grok, ChatGPT, OpenRouter, or any self‑hosted OpenAI‑compatible server. No mandatory sign‑in; you only need a reachable endpoint and the appropriate API key.
• Zero‑install runtime – a pre‑built binary (macOS Apple Silicon, Intel, Linux x86_64/ARM64) can be installed with a single curl … | sh command. No Rust toolchain is required for end‑users.
• Profile‑based configuration – settings live under ~/.distill (or a custom DISTILL_HOME). You can keep separate credentials for each provider and switch models with simple slash commands (/model, /worker-model, /utility-model).
• Live updates – distill update upgrades the installed binary while preserving your config, credentials and session state.

How to get started

# Install the binary (macOS/Linux)
curl -fsSL https://raw.githubusercontent.com/samuelfaj/distill/main/install.sh -o /tmp/distill-install.sh
sh /tmp/distill-install.sh
export PATH="$HOME/.local/share/distill/bin:$PATH"

distill            # launch the interactive UI

Inside the UI you can change models via the Model tiers screen or by typing commands such as:

/model gpt-6-astra auto
/worker-model gpt-5.6-luna auto
/utility-model openrouter-qwen37 auto

The tool will automatically verify that the chosen worker fits within the reasoning model’s context window.

Running with a local model – start any OpenAI‑compatible server (e.g., Ollama, vLLM) and add an entry to ~/.distill/config.toml:

[models]
default = "local"

[model.local]
name = "Local model"
model = "my‑local‑model"
base_url = "http://127.0.0.1:8000/v1"
api_backend = "chat_completions"

Distill will then route all tiers to that server (or you can mix a local utility model with a remote reasoning model).

Building from source – if you need a custom binary, clone the repo and compile with Cargo:

git clone https://github.com/samuelfaj/distill.git
cd distill
cargo build --locked --release -p distill-pager-bin --bin distill
./target/release/distill

The repository includes a rust-toolchain.toml file that pins the required Rust version.

License – Apache 2.0 (see LICENSE). Contributions are welcomed under the policy in CONTRIBUTING.md.

---

Distill is essentially a terminal‑first LLM‑powered pair programmer, letting developers stay inside their shell while the model reads, edits, and runs code on their behalf. It abstracts away provider details, offers automatic model‑tier routing, and works with both cloud APIs and self‑hosted models.