gpustack/gguf-parser-go

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

What it solves

GGUF Parser allows you to inspect GGUF model files and estimate their hardware requirements without needing to download the entire file or run the model. It helps users quickly determine if a model will fit in their system's memory and how it might perform.

How it works

The tool uses chunked reading to parse metadata directly from local files or remote URLs (including HuggingFace, ModelScope, and Ollama). By reading only the necessary parts of the file, it avoids full downloads. It then calculates memory usage (RAM/VRAM) and estimates performance metrics like tokens per second based on your specific hardware configuration.

Who it’s for

This tool is designed for developers and AI engineers who work with GGUF-formatted models and need a fast, efficient way to verify model compatibility and resource allocation before deployment.

Highlights

  • No Full Download Required: Uses chunked reading to inspect remote files without downloading the entire model.
  • Accurate Memory Estimation: Predicts memory usage with high precision, typically deviating by only about 100MiB.
  • Performance Forecasting: Estimates maximum tokens per second based on provided device metrics.
  • Versatile Compatibility: Supports various model types including Embedding, Reranking, and Decision models, as well as specialized architectures like MLA and SWA.
  • Efficient Performance: Written in Go for fast, lightweight execution.

Related

  • Project
    alvarobartt/hf-mem

    hf‑mem is a lightweight Python CLI (and library) that estimates the RAM required to run any Hugging Face model that uses Safetensors or GGUF weights. It fetches only metadata via HTTP range requests, supports Transformers, Diffusers, Sentence‑Transformers, and can optionally estimate KV‑cache memory and MoE breakdowns. Installable as a standalone tool, a Hugging Face CLI extension, or an agent skill.

  • Dispatch
    Beyond the Weights: Understanding GGUF and the Future of Model Metadata

    An exploration of the GGUF file format, its advantages in consolidating model metadata, and the critical gaps that still need to be filled for seamless LLM deployment.

  • Dispatch
    Hugging Face Transformers GGUF Support

    Hugging Face has added support for running GGUF quantized models directly in the transformers library, enabling efficient local inference on Apple Silicon using ggml kernels.

  • Project
    bodaay/HuggingFaceModelDownloader

    hfdownloader is a Go‑based CLI (and optional web UI) that speeds up HuggingFace model downloads with parallel chunking, interactive GGUF selection, full proxy support, and cache‑sync features, while storing files in the standard HuggingFace cache for seamless use with Python libraries.

  • Project
    gufo-org/gufo

    Gufo is an open‑source, AMD‑Strix Halo‑optimised inference engine that serves large language, vision and audio models via an OpenAI‑compatible API, with containerised deployment and speculative decoding for high throughput.