DaoyuanLi2816/can-i-finetune-this

Single-GPU LLM fine-tuning preflight: memory estimates, runnable LoRA/QLoRA recipes, and local measurements.

What it solves

It helps users determine if a specific Large Language Model (LLM) can be fine-tuned on their available GPU VRAM without having to download weights or load the model into memory first. It removes the guesswork from planning training runs by providing memory budget estimates and configuration suggestions to avoid Out-of-Memory (OOM) errors.

How it works

The tool uses a Python-based estimator that calculates the memory required for weights, quantization, trainable gradients, optimizer states, activations, and logits/loss buffers. It can generate runnable training recipes based on these estimates. Users can then run these recipes to measure actual local training performance and compare the observed memory peaks against the predicted budget.

Who it’s for

Developers and AI researchers using consumer NVIDIA GPUs who want to plan their LLM fine-tuning configurations (such as sequence length, batch size, and LoRA rank) before committing to a training run.

Highlights

  • No-weight estimation: Core memory estimation requires no PyTorch and no model-weight downloads.
  • Cofiguration planning: Suggests sequence lengths, batch sizes, and LoRA ranks based on a VRAM budget.
  • Runnable recipes: Generates executable training scripts using Transformers and PEFT.
  • Interactive demo: Includes a local web-based entry point for inspecting memory breakdowns.
  • Evidence-based: Allows users to record and compare actual measured memory peaks against predictions.

Related

  • Dispatch
    Optimizing Local LLM Selection: An In-Depth Look at whichllm

    Explore how whichllm solves the challenge of finding the best-performing local LLM for specific hardware using evidence-based ranking and VRAM estimation.

  • Project
    AlexsJones/llmfit

    llmfit is a cross‑platform CLI/TUI (and optional web/API) tool that auto‑detects your CPU, RAM, GPU and accelerator setup, then ranks hundreds of open‑source LLMs (including various quantizations) by how well they fit, their estimated speed, quality and context length. It lets you benchmark models on your own hardware, share results, and consume recommendations via JSON or a REST API, making it easy to pick and deploy a model that actually runs on your machine.

  • Project
    aidatatools/ollama-benchmark

    A CLI tool that benchmarks the token‑per‑second throughput of locally‑run Ollama models, automatically selecting models based on available RAM and reporting performance.

  • Project
    unslothai/unsloth-zoo

    A collection of utilities and optimized kernels for Unsloth that enables faster and more memory-efficient fine-tuning of LLMs and multimodal models with significantly reduced VRAM usage.

  • Project
    NVIDIA/TensorRT-LLM

    TensorRT‑LLM is NVIDIA’s open‑source library that compiles large language (and visual‑generation) models into ultra‑fast TensorRT inference engines. It provides custom CUDA kernels, quantization, sparse attention, speculative decoding, and multi‑GPU parallelism, delivering 2‑10× higher token throughput while keeping accuracy. The Python‑friendly runtime integrates with Triton, Kubernetes, and Jetson, and includes docs and examples for models like GPT‑OSS‑120B, Llama‑4, DeepSeek‑R1, and diffusion models.