alvarobartt/hf-mem

📦 Project: hf‑mem – Memory‑usage estimator for Hugging Face models

What it is

• A tiny Python‑only command‑line tool that tells you how much RAM (and KV‑cache) a model will need at inference time.
• Works for any model hosted on the Hugging Face Hub that stores its weights as Safetensors or GGUF files – this includes Transformers, Diffusers, Sentence‑Transformers and even custom GGUF models.
• Designed to be lightweight: the only runtime dependency is httpx2, and it fetches only the model metadata via HTTP Range requests, so it never downloads the full weight files.

Why it matters

• Before pulling a multi‑gigabyte model onto a device (CPU, GPU, or edge hardware) you can quickly check whether the available memory is sufficient.
• The optional experimental mode adds KV‑cache estimates for LLMs/VLMs and can break down memory for Mixture‑of‑Experts models, helping you size batch size, context length, and data‑type choices.

---

🚀 Quick start (CLI)

# Recommended: run with uv for fast, isolated execution
uvx hf-mem --model-id MiniMaxAI/MiniMax-M2          # Transformers model
uvx hf-mem --model-id Qwen/Qwen-Image               # Diffusers model
uvx hf-mem --model-id google/embeddinggemma-300m    # Sentence‑Transformers model

The command prints a table with the model ID, estimated model‑weight memory, KV‑cache memory (if --experimental), and total RAM needed.

🐍 Using it from Python

from hf_mem import run, arun

# Synchronous call
result = run(model_id="MiniMaxAI/MiniMax-M2", experimental=True)
print(result)

# Asynchronous call (inside an async app)
# result = await arun(model_id="MiniMaxAI/MiniMax-M2", experimental=True)

The returned Result object contains fields such as memory, kv_cache, and total_memory (all in bytes).

---

⚙️ Experimental extensions

• --experimental enables:
  • KV‑cache size estimation for causal LM and conditional‑generation models (you can tweak --max-model-len, --batch-size, --kv-cache-dtype).
  • A breakdown of base‑model vs. expert weights for Mixture‑of‑Experts architectures.
• GGUF support:
  • If a repo only contains GGUF files, each file is listed and its memory is estimated.
  • You can target a single GGUF file with --gguf-file <filename>.

---

🛠️ Installation & integration

1. Standalone – the package can be installed from PyPI (or directly from the repo) and run via uvx hf-mem ….
2. Hugging Face CLI extension – after installing the regular hf CLI, add the tool as an extension:

   hf extensions add alvarobartt/hf-mem
   

   Then invoke it as hf mem … alongside other HF extensions.
3. Agent skill – a SKILL.md is provided so coding agents (e.g., Anthropic Claude) can discover and call hf‑mem automatically.

---

📚 What it does not do

• It does not download or run the model; it only inspects the weight‑file headers.
• It does not perform benchmark‑style latency or throughput measurements.
• The experimental KV‑cache calculations are approximations and may change before the 1.0 release.

---

📖 Further reading

• Short‑form blog post (Jan 2026) – https://alvarobartt.com/hf-mem (may be slightly outdated).
• Safetensors metadata spec, GGUF file format, and Hugging Face Hub documentation are linked in the README for deeper technical details.

---

Bottom line: hf‑mem gives developers a fast, dependency‑light way to size the RAM needed for any Hugging Face model before pulling it, helping avoid out‑of‑memory surprises on local machines, servers, or edge devices.