llmfit: a hardware-aware model recommender that matches LLMs to your system's RAM and GPU

What it solves

It solves the difficulty of choosing a Large Language Model (LLM) that is compatible with a user's specific hardware. Instead of guessing if a model will fit in VRAM or run at an acceptable speed, users can find models that are right-sized for their available RAM, CPU, and GPU.

How it works

llmfit detects the system's hardware specifications and scores a catalog of hundreds of models across four key dimensions: memory fit, estimated speed, quality, and context. Speed estimates are derived from a memory-bandwidth model based on runtime sampling and community-contributed measurements. The tool provides an interactive TUI (Terminal User Interface) and a CLI for automation, supporting various local runtime providers like Ollama, llama.cpp, MLX, and LM Studio.

Who it’s for

It is designed for developers and AI enthusiasts running local LLMs who want to optimize model selection based on their actual hardware constraints without manual calculations.

Highlights

• Hardware Detection: Automatically identifies RAM, CPU, and GPU/VRAM to determine model compatibility.
• Multi-Backend Support: Compatible with local providers including Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio.
• Community Benchmarking: Allows users to measure real tokens-per-second on their hardware and contribute results back to the project to improve estimates for others.
• Flexible Interface: Offers both a visual TUI for exploration and a JSON-output CLI for integration into scripts or agents.
• Architecture Awareness: Supports complex model types, including Mixture-of-Experts (MoE) architectures.