aidatatools/ollama-benchmark

LLM Benchmark for Throughput via Ollama (Local LLMs)

What it solves

It gives you a concrete number for how fast your local large language models actually run. Instead of guessing or relying on provider claims, you get a tokens-per-second throughput measurement for your own hardware.

How it works

You install it with pip, pipx, or uv, then run a single command. It relies on an existing Ollama installation. The tool detects your system's RAM size, automatically chooses a set of suitable models to benchmark, pulls them if they aren't already present, and runs prompt workloads to measure throughput. You can also provide a custom YAML file with your own model list. By default it sends system information and results to a remote server; use the --no-sendinfo flag to keep everything local.

Who it’s for

Anyone running local LLMs through Ollama — developers, hobbyists, researchers — who wants a quick, repeatable way to compare inference speed on their own machine.

Highlights

  • Cross-platform: works on Windows, Linux, and macOS.
  • Automatically selects models based on RAM size.
  • Can optionally send benchmark results to a remote server.
  • Supports custom benchmark model lists via YAML files.
  • Tested on Python 3.10 and above.

Related

  • Project
  • Project
  • Project
  • Project
  • Project