gpustack/gguf-parser-go

What it solves

GGUF Parser is a tool for reviewing and estimating the resource requirements of models in the GGUF format without needing to download the entire file. It helps users determine if a model will fit in their hardware (RAM/VRAM) and predicts performance metrics like tokens per second (TPS) before deployment.

How it works

Written in Go for efficiency, the tool uses chunked reading to extract metadata from local or remote GGUF files (including those hosted on HuggingFace, ModelScope, and Ollama). It analyzes the model's architecture, quantization, and tokenizer settings to calculate estimated memory usage across different hardware configurations, including support for multi-GPU setups, RPC servers, and Apple Silicon (UMA).

Who it’s for

It is designed for developers and AI researchers who use GGML-based executors and need to verify model specifications and memory footprints quickly without wasting bandwidth or storage on incompatible models.

Highlights

• Remote Parsing: Analyze metadata from URLs without downloading the full model file.
• Accurate Memory Estimation: Predicts RAM and VRAM usage with a typical deviation of approximately 100MiB.
• Performance Prediction: Estimates maximum tokens per second (TPS) based on provided device metrics.
• Broad Model Support: Compatible with various architectures including DeepSeek (MLA), LLaMA 4, Gemma 2/3 (SWA), and audio projector models (Ultravox, Qwen2 Audio).
• Type Screening: Identifies if a GGUF file is intended for Embedding, Reranking, LoRA, or image generation (StableDiffusion.Cpp).