unslothai/unsloth

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

What it solves

Unsloth is a comprehensive toolset for running and training AI models locally. It addresses the high hardware requirements and slow speeds associated with fine-tuning and deploying large language models (LLMs) and other AI models on consumer-grade hardware.

How it works

Unsloth provides a desktop application, a web UI (Unsloth Studio), and a code-based version (Unsloth Core). It optimizes the training process, allowing users to fine-tune models 2x faster with 70% less VRAM without losing accuracy. It supports a wide range of hardware (NVIDIA, AMD, Intel GPUs, CPUs) and formats (GGUF, MLX, FP8).

Who it’s for

It is designed for developers, AI researchers, and enthusiasts who want to run, fine-tune, and deploy AI models locally on Windows, macOS, and Linux, including those using consumer GPUs.

Highlights

  • High-Performance Training: Fine-tuning for LLMs, diffusion, TTS, and embedding models with significant VRAM reduction and speed increases.
  • Broad Hardware Support: Compatible with Multi-GPU setups, NVIDIA, AMD, Intel GPUs, CPUs, and the Vulkan backend.
  • Local Agent Integration: The unsloth start command connects local models to agents like Claude Code, Codex, and others via OpenAI- and Anthropic-compatible APIs.
  • Local RAG and Search: Includes private web search, deep research, and RAG capabilities.
  • Versatile Deployment: Supports exporting models to GGUF, NVFP4, and FP8 formats.
  • Easy Installation: Available as a native desktop app, Docker image, or via shell scripts for various operating systems.

Related

  • Project
    unslothai/unsloth-zoo

    A collection of utilities and optimized kernels for Unsloth that enables faster and more memory-efficient fine-tuning of LLMs and multimodal models with significantly reduced VRAM usage.

  • Dispatch
    Unsloth and Hugging Face TRL Integration for Faster LLM Fine-tuning

    Unsloth is a lightweight library that accelerates LLM fine-tuning by up to 2.7x and reduces memory usage by up to 74% with 0% accuracy degradation compared to QLoRA.

  • Project
    ARahim3/mlx-tune

    A library for fine-tuning LLMs, vision, audio, and OCR models natively on Apple Silicon using an Unsloth-compatible API for easy prototyping and scaling to cloud GPUs.

  • Project
    unslothai/notebooks

    A collection of Google Colab notebooks that demonstrate how to fine‑tune, run inference and experiment with reinforcement‑learning on many popular LLMs (Gemma, Qwen, Llama 3, Mistral, Phi‑4, etc.) using the Unsloth library. Each notebook is self‑contained, installs the required packages, downloads the model, prepares data, trains and then shows inference, making cutting‑edge LLM work accessible without local GPU setup.

  • Dispatch
    Train AI models with Unsloth and Hugging Face Jobs

    Hugging Face has integrated Unsloth with Hugging Face Jobs to enable fast, low-cost LLM fine-tuning, specifically optimized for small models like LiquidAI/LFM2.5-1.2B-Instruct.