unslothai/unsloth-zoo

Utils for Unsloth https://github.com/unslothai/unsloth

What it solves

Unsloth Zoo provides a set of utilities and optimized kernels to make fine-tuning Large Language Models (LLMs) significantly faster and more memory-efficient. It addresses the high VRAM requirements and slow training speeds typically associated with fine-tuning modern transformer-style models.

How it works

The project uses custom kernels written in OpenAI's Triton language and a manual backpropagation engine to optimize the training process. This allows for faster training (up to 2.2x) and drastically reduced VRAM usage (up to 80% less) without any loss in accuracy. It supports various training modes, including full-finetuning, 4-bit, 8-bit, and 16-bit training, and is compatible with NVIDIA GPUs from 2018 onwards.

Who it’s for

It is designed for developers and AI researchers who want to fine-tune LLMs (such as Llama, Mistral, Qwen, and Gemma) on their own hardware or in free notebooks (like Colab) without needing massive GPU clusters.

Highlights

  • Extreme Efficiency: Reduces VRAM usage by 50-80% and increases training speed by 1.5x to 2.2x.
  • Broad Model Support: Compatible with all transformer-style models, including text LLMs, multimodal vision models, and Text-to-Speech (TTS) models.
  • Zero Accuracy Loss: Uses exact calculations rather than approximation methods.
  • Flexible Export: Supports exporting fine-tuned models to GGUF, Ollama, vLLM, or Hugging Face.

Related

  • Dispatch
    Unsloth and Hugging Face TRL Integration for Faster LLM Fine-tuning

    Unsloth is a lightweight library that accelerates LLM fine-tuning by up to 2.7x and reduces memory usage by up to 74% with 0% accuracy degradation compared to QLoRA.

  • Project
    unslothai/unsloth

    A desktop app and toolkit for running and training AI models locally, offering significantly faster fine-tuning with reduced VRAM usage on consumer hardware.

  • Project
    ServiceNow/Fast-LLM

    A high-performance library for training large language models that uses 3D parallelism and optimized kernels to reduce training time and cost.

  • Project
    unslothai/notebooks

    A collection of ready-to-use Google Colab notebooks for fine-tuning, training, and running inference on various open-source LLMs and multimodal models.

  • Project
    ARahim3/mlx-tune

    A library for fine-tuning LLMs, vision, audio, and OCR models natively on Apple Silicon using an Unsloth-compatible API for easy prototyping and scaling to cloud GPUs.