DaoyuanLi2816/can-i-finetune-this

Estimate whether a Hugging Face model fits and fine-tunes on your local GPU.

What it solves

It prevents users from wasting time and disk space downloading large LLM weights only to find out their consumer GPU lacks the VRAM required to fine-tune the model. It specifically addresses the gap between knowing how much memory is needed to load a model versus how much is needed to train it.

How it works

The tool uses a memory model to statically estimate VRAM usage based on model weights, LoRA/QLoRA trainable parameters, gradients, optimizer states, and activations (which vary by sequence length and batch size). To improve accuracy, it allows users to run local benchmarks on their own hardware and calibrate the estimates to account for machine-specific overheads like CUDA fragmentation and buffer unpacking.

Who it’s for

Developers and AI enthusiasts using consumer‑grade NVIDIA GPUs (typically 12–24 GB VRAM) who want to fine‑tune open‑weight LLMs using the Hugging Face, PEFT, and TRL ecosystem.

Highlights

  • VRAM Estimation: Predicts memory usage for various quantization levels (fp32, fp16, bf16, int8, NF4) and training methods.
  • Feasibility Checks: Determines if a specific model and configuration will fit on a given GPU and suggests downsizing options if it doesn't.
  • Local Benchmarking: Includes a bench command to run small‑scale tests to ground static estimates in real‑world hardware evidence.
  • Recipe Generation: Automatically generates ready‑to‑run Hugging Face + PEFT + TRL training scripts based on the recommended configuration.