AlexsJones/llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

llmfit – What it does

llmfit is a command‑line (and optional web) tool that helps you figure out which open‑source Large Language Models (LLMs) can actually run on your computer. It automatically detects your CPU, RAM, GPUs, VRAM and other accelerators, then scores every model in its built‑in catalog on:

  • Fit – will the model fit into memory?
  • Speed – estimated tokens‑per‑second based on a bandwidth model and community benchmarks.
  • Quality – a rough quality ranking supplied by the catalog.
  • Context – maximum prompt length the model supports.

The result is a ranked list of models (including various quantizations such as GGUF, AWQ, GPTQ, EXL2) that you can run locally with back‑ends like Ollama, llama.cpp, MLX, Docker runners or LM Studio.

Key features (from the README)

Feature What you get
Hardware auto‑detection CPU cores, system RAM, discrete/integrated GPUs, VRAM, and unified memory (CUDA, Apple Silicon, ROCm, OneAPI).
Model compatibility engine Projects memory footprints and token‑per‑second performance for many quantization formats.
Interactive TUI & web dashboard A zero‑dependency terminal UI (default) or a richer web UI that can be run locally or in Docker.
REST API JSON endpoints (/api/v1/system, /api/v1/models) for integration with orchestration tools or custom dashboards.
Cross‑platform macOS (Apple Silicon & Intel), Linux (x86_64 & ARM64), Windows (x86_64).
Community benchmarking Run a real benchmark on your hardware, share the results with a single command, and have them incorporated into future releases.
Container image Multi‑arch Docker image (ghcr.io/alexsjones/llmfit) that can serve the TUI, CLI, or API.
Extensible model catalog Add new models locally or via PR; the catalog already contains hundreds of models from many providers.

Typical workflow

  1. Install – via Homebrew, Scoop, a quick‑install script, a pre‑built binary, or as a Rust/Python package.
  2. Run – llmfit (or llmfit recommend) to see a table of compatible models for your detected hardware.
  3. Explore – Use the TUI to filter, sort, view detailed fit analysis, or launch the web UI.
  4. Benchmark – llmfit bench measures real token‑per‑second speed for a chosen model/provider and can --share the numbers back to the project.
  5. Deploy – Pick a model, download it, and start it with your preferred local runtime (e.g., Ollama, llama.cpp).
  6. Automate – Consume JSON output (llmfit recommend --json) or call the REST API for scripted deployments.

Who might use it?

  • Developers who want to run LLMs locally but are unsure which size/quantization will fit.
  • Researchers evaluating hardware for LLM workloads.
  • Ops / DevOps building automated pipelines that need to select a model based on the target machine.
  • Enthusiasts contributing benchmark data to improve the community’s estimates.

Installation highlights

  • Windows – scoop install llmfit
  • macOS / Linux – Homebrew (brew install AlexsJones/llmfit/llmfit), MacPorts, or the one‑liner curl … | sh
  • Python / uv – uv tool install -U llmfit (also works with pip)
  • Docker – docker run ghcr.io/alexsjones/llmfit (add --tui for interactive mode or serve for the API/UI)
  • From source – cargo build --release

License & signing

The project is released under the MIT license. Windows binaries are signed via SignPath when the CI job succeeds; unsigned binaries may still be published if signing fails.

Quick start example (CLI)

# Show hardware profile and top‑fit models
llmfit recommend

# Get the same info as JSON for a script
llmfit recommend --json > fits.json

# Run a real benchmark on a model and share the result
llmfit bench --model "Llama-2-7B-Chat-GGUF" --provider ollama --share

In short: llmfit is a practical utility that bridges the gap between the growing catalog of open‑source LLMs and the diverse hardware people run them on, offering both estimates and real‑world measurements to help you pick the right model.

Related

  • Project
    midudev/canirun.ai

    CanIRun.ai is a browser-based tool that automatically detects your CPU, RAM, and GPU using standard web APIs, then recommends the best open-weight AI models to run locally for coding, chat, reasoning, and vision tasks. It grades each model S–F based on your hardware, with no installs or benchmarks needed.

  • Project
    signerless/llm-checker

    An intelligent CLI tool that analyzes your hardware to recommend, rank, and manage local LLMs, featuring hardware-calibrated memory estimation and an integrated MCP server for AI assistants.

  • Dispatch
    Optimizing Local LLM Selection: An In-Depth Look at whichllm

    Explore how whichllm solves the challenge of finding the best-performing local LLM for specific hardware using evidence-based ranking and VRAM estimation.

  • Project
    Andyyyy64/whichllm

    whichllm is a CLI that detects your hardware (or a simulated GPU) and ranks Hugging Face LLMs that will actually run locally, using live benchmark data, memory‑fit estimates and speed predictions. It can output tables, Markdown, JSON, start a chat, or generate Python snippets, and includes planning commands for hardware upgrades.

  • Project
    pytorch/cpuinfo

    A cross-platform C/C++ library for detecting detailed CPU hardware information, such as instruction sets and cache topology, to enable performance optimizations.