unslothai/unsloth-zoo

Utils for Unsloth https://github.com/unslothai/unsloth

What it solves

Unsloth Zoo provides utilities to make the fine-tuning of large language models (LLMs) significantly faster and more memory-efficient. It addresses the high VRAM requirements and slow training speeds typically associated with fine-tuning frontier models like Llama, Mistral, and Qwen.

How it works

The project uses custom kernels written in OpenAI's Triton language and a manual backpropagation engine to optimize training. It supports various training modes including full-finetuning, 4-bit, 8-bit, and 16-bit training without sacrificing accuracy. It is designed to work on NVIDIA GPUs (2018+) and integrates with Hugging Face's TRL and Trainer libraries.

Who it’s for

It is designed for developers and AI researchers who want to fine-tune LLMs, vision models, or text-to-speech (TTS) models on limited hardware (such as consumer GPUs) or who need to accelerate their training pipelines.

Highlights

  • Significant Performance Gains: Up to 2.2x faster training with up to 80% less VRAM usage.
  • Broad Model Support: Compatible with Llama 3.1/3.2/4, Mistral, Qwen3, Gemma 3, Phi-4, and others, including vision and TTS models.
  • Zero Accuracy Loss: Uses exact calculations rather than approximation methods.
  • Flexible Export: Allows exporting fine-tuned models to GGUF, Ollama, vLLM, or Hugging Face formats.
  • Advanced RL Support: Includes memory-efficient Reinforcement Learning (RL) and GRPO for training reasoning models.

Related

  • Dispatch
    Unsloth and Hugging Face TRL Integration for Faster LLM Fine-tuning

    Unsloth is a lightweight library that accelerates LLM fine-tuning by up to 2.7x and reduces memory usage by up to 74% with 0% accuracy degradation compared to QLoRA.

  • Project
    unslothai/unsloth

    A desktop app and framework for running and training AI models locally, offering significantly faster fine-tuning with reduced VRAM usage across multiple hardware platforms.

  • Project
    unslothai/notebooks

    A collection of Google Colab notebooks that demonstrate how to fine‑tune, run inference and experiment with many open‑source LLMs (Gemma, Llama 3, Qwen, Mistral, Phi‑4, etc.) using the Unsloth library. Includes chat UI, vision/audio/embedding demos, and reinforcement‑learning/GRPO training notebooks, all runnable for free in the browser.

  • Project
    ServiceNow/Fast-LLM

    A high-performance library for training large language models that uses 3D parallelism and optimized kernels to reduce training time and cost.

  • Project
    DaoyuanLi2816/can-i-finetune-this

    A CLI tool to estimate VRAM usage, benchmark performance, and generate training recipes for fine-tuning LLMs on consumer GPUs using LoRA and QLoRA.