aidatatools/ollama-benchmark

LLM Benchmark for Throughput via Ollama (Local LLMs)

What it solves

This tool provides a way to measure the actual throughput (tokens-per-second) of local Large Language Models (LLMs) running via Ollama, allowing users to understand how their hardware performs with different model sizes.

How it works

The CLI tool interacts with an Ollama installation. It automatically detects the system's RAM size to determine which LLM models are appropriate to download and test. It then runs benchmarks against these models to calculate performance metrics.

Who it’s for

Users running local LLMs through Ollama who want to quantify their hardware's inference speed across various model tiers (from 2B to 20B parameters).

Highlights

  • Cross-platform CLI tool for Windows, Linux, and macOS.
  • Automatic model selection based on available system RAM.
  • Support for custom benchmark model lists via YAML files.
  • Option to disable the transmission of system info and results to a remote server.

Related

  • Project
    TensorOpsAI/LLMstudio

    A prompt engineering platform that provides a unified proxy for multiple LLM providers and local models, featuring a playground UI and monitoring tools.

  • Project
    vueuse/skills

    A collection of agent skills that enables AI agents to use the VueUse library more accurately and efficiently by reducing token consumption and hallucinations.

  • Project
    cisco-open/network-sketcher

    An AI-native network design tool that generates L1/L2/L3 topology diagrams and device tables from structured data or natural language workflows.

  • Project
    rodrigo-arenas/Sklearn-genetic-opt

    A scikit-learn-compatible optimization toolkit that uses genetic algorithms for hyperparameter tuning and feature selection, offering a more efficient alternative to grid and random search.