aidatatools/ollama-benchmark
LLM Benchmark for Throughput via Ollama (Local LLMs)
What it solves
It gives you a concrete number for how fast your local large language models actually run. Instead of guessing or relying on provider claims, you get a tokens-per-second throughput measurement for your own hardware.
How it works
You install it with pip, pipx, or uv, then run a single command. It relies on an existing Ollama installation. The tool detects your system's RAM size, automatically chooses a set of suitable models to benchmark, pulls them if they aren't already present, and runs prompt workloads to measure throughput. You can also provide a custom YAML file with your own model list. By default it sends system information and results to a remote server; use the --no-sendinfo flag to keep everything local.
Who it’s for
Anyone running local LLMs through Ollama — developers, hobbyists, researchers — who wants a quick, repeatable way to compare inference speed on their own machine.
Highlights
- Cross-platform: works on Windows, Linux, and macOS.
- Automatically selects models based on RAM size.
- Can optionally send benchmark results to a remote server.
- Supports custom benchmark model lists via YAML files.
- Tested on Python 3.10 and above.
Related
- Project
signerless/llm-checkerAn intelligent CLI tool that analyzes your hardware to recommend, rank, and manage local LLMs, featuring hardware-calibrated memory estimation and an integrated MCP server for AI assistants.
- Project
sammcj/gollamaGollama is a Go‑based terminal UI for listing, inspecting, editing, running, unloading, copying, deleting and pushing Ollama models, with command‑line shortcuts and a vRAM estimator to help decide if a model fits on your hardware.
- Project
AlexsJones/llmfitllmfit is a hardware-aware tool that inspects your system’s CPU, RAM, GPU, and VRAM, then scores hundreds of open-source LLMs across fit, speed, quality, and context to tell you which models will actually run well on your machine. It includes an interactive TUI, a web dashboard, and a REST API, plus community benchmark sharing to refine estimates.
- Project
huggingface/optimum-benchmarkOptimum-Benchmark is a Hugging Face tool that measures AI model performance (speed, memory, energy) across different hardware and software backends. It supports inference and training scenarios, runs on CPU, NVIDIA, AMD, and other devices, and can be used via a Python API or command-line interface. It's under active development and not yet production-ready.
- Project
taketwo/llm-ollamallm-ollama is a plugin for the LLM command-line tool that connects to an Ollama server (local, remote, or cloud) and automatically registers all available models, enabling prompting, chatting, embeddings, attachments, tools, and more directly through LLM's interface.