AlexsJones/llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

What it solves

llmfit helps users determine which open-source Large Language Models (LLMs) their specific hardware can run comfortably. It removes the guesswork from selecting a model by analyzing system resources and recommending models based on available memory and expected performance.

How it works

The tool detects your system's CPU, RAM, GPU, VRAM, and accelerator configuration (supporting NVIDIA CUDA, Apple Silicon, AMD ROCm, and Intel OneAPI). It then compares these specs against a catalog of models, analyzing parameter counts, context lengths, and quantization formats (such as GGUF, AWQ, GPTQ, and EXL2).

Performance is estimated using a memory-bandwidth model grounded in runtime sampling and community-contributed benchmarks. Users can also run their own benchmarks to replace estimates with real-world tokens-per-second measurements on their specific hardware.

Who it’s for

Developers and AI enthusiasts who want to run local LLMs but aren't sure which model size or quantization level will fit in their VRAM/RAM without crashing their system.

Highlights

  • Hardware Auto-Detection: Automatically identifies system resources across macOS, Linux, and Windows.
  • Interactive TUI & Web Dashboard: Offers a terminal-based interface or a web-based dashboard for browsing and ranking models.
  • Community Benchmarking: Allows users to measure real performance and contribute results back to the project to improve estimates for others with identical hardware.
  • REST API: Provides JSON endpoints for integration into other deployment pipelines or orchestrators.
  • Broad Provider Support: Works with local runtimes like Ollama, llama.cpp, MLX, and LM Studio.

Related

  • Project
    midudev/canirun.ai

    A hardware detection and recommendation tool that tells users which open-weight AI models will run best on their specific machine based on their CPU, RAM, and GPU.

  • Project
    signerless/llm-checker

    An AI-powered CLI tool that analyzes hardware and recommends optimal local LLM models from a multi-source registry, featuring integrated safety verification and MCP support.

  • Project
    Andyyyy64/whichllm

    A hardware-aware LLM recommendation tool that ranks the best HuggingFace models based on your specific GPU/CPU/RAM and real-world benchmark performance.

  • Dispatch
    Optimizing Local LLM Selection: An In-Depth Look at whichllm

    Explore how whichllm solves the challenge of finding the best-performing local LLM for specific hardware using evidence-based ranking and VRAM estimation.

  • Project
    AlexsJones/llmserve

    A TUI-based model manager and server launcher that auto-detects inference backends and local model files to quickly spin up LLM servers.