ThorOdinson246/whatisit-nl2sh

whatisit‑nl2sh – Turn plain English into a shell command (offline)

What it does – whatisit is a small command‑line tool that takes a natural‑language description of a task and returns the corresponding Unix shell command. It runs a 1.5 B‑parameter LLM locally via llama.cpp, so no data ever leaves your machine unless you explicitly point it at a remote OpenAI‑compatible endpoint.

Why it matters – It lets you get quick, correct‑looking commands without searching the web or remembering exact syntax, while keeping privacy and avoiding the latency of cloud APIs. The model is tiny (≈ 941 MiB) and can run on a CPU‑only laptop.

---

Quick start

# Install the CLI (pipx is recommended but plain pip works too)
pipx install whatisit          # or: pip install whatisit
whatisit setup                 # downloads the model and checks your llama.cpp binary

If you already have a llama.cpp build, point whatisit at it with --bin-dir.

Typical usage

# Simple query – prints the command
whatisit find files bigger than 100MB in this folder
# → find . -size +100M -exec ls -lh {} \;

# Run the command after confirming it
whatisit -e compress the logs directory into a tarball

# Get several alternatives
whatisit -n 3 list python files modified today

# Use the output in a script
cd "$(whatisit -q the directory holding the largest log file)"

Flags:

• -e/--execute – ask for confirmation then run the command.
• -n N – return N candidate commands.
• -q/--quiet – output only the raw command (useful for command substitution).
• -t/--timing – show how long generation took.
• -p/--prefill-command – open the generated command in an editable prompt before running.

You can also integrate it into your shell prompt:

# Add to ~/.bashrc or ~/.zshrc
eval "$(whatisit shell-init)"

Now whatisit <request> places the generated command on the next prompt line for easy editing.

---

How it works under the hood

1. Model – A LoRA‑fine‑tuned version of Qwen2.5‑Coder‑1.5B‑Instruct (or an optional 3 B variant) quantised to GGUF Q4_K_M (≈ 941 MiB). 125 k NL‑to‑command pairs were used for training.
2. Runtime – The model is served by llama.cpp’s llama-server. The first call starts a resident server; subsequent calls are fast because the model stays loaded in RAM (≈ 1.6 GiB for the 1.5 B model).
3. Generation – Greedy decoding at temperature 0, so the same query always yields the same command. Typical throughput on an Intel i5‑11320H: ~39 tokens / s, ~0.6 s latency after warm‑up.
4. Safety layer – Before printing, a rule‑based checker scans the command for dangerous patterns (recursive deletes of /, writes to block devices, privilege‑escalation tricks, reverse shells, etc.). Flagged commands are labeled DANGER (never auto‑run) or CAUTION (warned but still optional).

---

Offline vs. remote mode

• Default – Fully offline; the request never leaves the host.
• Remote – Set openai_base_url and openai_model (or an Ollama endpoint) in the config to route queries to a hosted API. The tool prints a warning on every remote call.

---

Installation details

Step	What you need
CLI	pipx install whatisit (or pip install whatisit in a venv)
Model file	Download nl2sh-1.5b-Q4_K_M.gguf (≈ 941 MiB) from Hugging Face via huggingface_hub
llama.cpp binary	Grab a pre‑built release (llama-server, llama-cli) or build it yourself; glibc ≥ 2.34 required on Linux
Configuration	whatisit setup --model <path> --bin-dir <path/to/llama.cpp/bin>

A Nix flake is also provided for users of Nix/NixOS, wiring in the llama.cpp package automatically.

---

Performance & benchmarks

• InterCode‑ALFA benchmark (300 NL‑to‑shell tasks)
  • whatisit (1.5 B) – 62 % pass rate, 0.59 s median latency (warm), 2 s cold start.
  • Optional 3 B model – 65.7 % pass, slower generation (≈ 17 tok/s) and higher RAM (~3.4 GiB).
  • Compared to untuned Qwen2.5‑Coder‑7B (61.3 %) and GPT‑4o (73 %).
• The improvement over the untuned base (54 %) is statistically significant (p ≈ 0.004).

---

Safety & limitations

• Safety – Rule‑based denylist covering recursive deletes, block‑device writes, privilege changes, network‑pull commands, etc. It is not a sandbox; you must still review output.
• Limitations – Single‑turn only (no shell state), max 64‑token output (so not suitable for multi‑line scripts), English‑only, benchmark‑specific performance.

---

License & attribution

• Code: Apache‑2.0
• Model: derived from Qwen2.5‑Coder‑1.5B‑Instruct (Apache‑2.0) with training data from various permissively‑licensed sources (MIT, CC‑BY‑4.0, CC0, Apache‑2.0). Full dataset to be released later.

---

Where to go next

• Try the 3 B model for higher accuracy (whatisit setup --size 3b).
• Contribute improvements to the safety checker or add support for multi‑turn interactions.
• Follow the repository for daily update checks (can be disabled via config).

---

Bottom line – whatisit gives you a privacy‑preserving, low‑resource way to translate everyday English instructions into shell commands, with a built‑in safety filter and optional remote‑API fallback.