hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

LlamaFactory – One‑click fine‑tuning for 100+ LLMs

What it is LlamaFactory is an open‑source Python library (also packaged on PyPI) that lets you fine‑tune a huge variety of large language models – from classic LLaMA‑style models to multimodal LLaVA, Mistral, Mixtral‑MoE, Qwen, Gemma, GLM, Phi, and many others. It provides a zero‑code CLI and a Gradio‑based web UI (LlamaBoard) so you can start training on a laptop, a cloud notebook, or a multi‑GPU cluster without writing Python scripts.

Why it matters Fine‑tuning modern LLMs usually requires fiddling with low‑level libraries (PEFT, DeepSpeed, FSDP, etc.). LlamaFactory hides that complexity and bundles a large collection of state‑of‑the‑art training tricks (LoRA, QLoRA, FlashAttention‑2, Unsloth, Liger‑Kernel, KTransformers, etc.) and optimizers (GaLore, BAdam, APOLLO, Muon, Adam‑mini, DoRA, LongLoRA, etc.). The result is a single configuration file that can train models ranging from a few million parameters up to 70 B‑scale on modest hardware.

Key features (as listed in the README)

  • Model coverage – LLaMA‑1/2/3/4, LLaVA, Mistral, Mixtral‑MoE, Qwen 2/3, Qwen‑VL, DeepSeek, Gemma 3, GLM‑4, Phi‑4, InternLM, MiniCPM‑o, PaliGemma, and many more.
  • Training methods – standard supervised fine‑tuning, continuous pre‑training, reward‑modeling, PPO, DPO, KTO, ORPO, and other preference‑learning algorithms.
  • Parameter‑efficient finetuning – full‑precision 16‑bit, freeze‑tuning, LoRA, and 2‑8‑bit QLoRA (AQLM, AWQ, GPTQ, LLM.int8, HQQ, EETQ).
  • Performance tricks – FlashAttention‑2, Unsloth, Liger‑Kernel, KTransformers, RoPE scaling, NEFTune, rsLoRA, and more.
  • Hardware flexibility – works on NVIDIA GPUs, AMD ROCm, and Ascend NPU; supports FSDP + QLoRA for 70 B models on 2 × 24 GB GPUs.
  • Inference back‑ends – OpenAI‑style REST API, Gradio UI, CLI, with optional vLLM or SGLang workers for high‑throughput serving.
  • Experiment tracking – built‑in support for TensorBoard, Weights & Biases, MLflow, and SwanLab.
  • Deployments – export checkpoints as Hugging Face, ModelScope, Modelers Hub, or Ollama modelfiles; run as an OpenAI‑compatible API container.
  • Community & ecosystem – used by Amazon, NVIDIA, Alibaba Cloud; has a Discord, WeChat groups, a dedicated blog, and a Docker image on Docker Hub.

Typical workflow

  1. Install (pip install llamafactory or pull the Docker image).
  2. Prepare data – the README points to a data-preparation section and provides example scripts for text, chat, tool‑call, image, video, and audio datasets.
  3. Write a tiny YAML config that selects the model, training method, LoRA/QLoRA settings, and any performance tricks you want.
  4. Run – either llamafactory train config.yaml (CLI) or launch the Gradio UI (llamafactory gui) to start training, monitor with LlamaBoard, and view logs.
  5. Export – after training, use llamafactory export to produce a model ready for inference, then serve it via the built‑in OpenAI‑compatible API or a vLLM worker.

Getting started resources

  • Quickstart section in the README shows a minimal CLI command.
  • Colab notebook (free GPU) and PAI‑DSW links for cloud‑based trials.
  • Documentation (WIP) at llamafactory.readthedocs.io and a performance‑comparison wiki.
  • Blog posts covering KTransformers, data‑centric pipelines, and multimodal fine‑tuning.

Who should use it

  • Researchers who need to experiment with many LLM families without rewriting training loops.
  • Product teams that want a code‑free UI for fine‑tuning and deploying custom assistants.
  • Hobbyists looking to fine‑tune a 7‑B model on a single GPU using LoRA/QLoRA.
  • Enterprises (Amazon, NVIDIA, Alibaba) that require a scalable, reproducible pipeline for large‑scale model adaptation.

Installation snippet (from the README)

# via pip
pip install llamafactory

# or pull the official Docker image
docker pull hiyouga/llamafactory

License – Apache 2.0 (as indicated in the License section).

All details above are taken directly from the repository’s README; no additional features have been invented.

Related

  • Project
    OptimalScale/LMFlow

    LMFlow is an extensible Python toolbox for fine‑tuning large language models. It supports full‑parameter training, LoRA, the memory‑efficient LISA algorithm, QLoRA quantization, and a suite of speed‑up tricks (FlashAttention‑2, DeepSpeed, vLLM, SGLang). After training, it provides simple scripts for chat‑style inference and optional Gradio/Flask deployment. The library is designed for Linux/Colab environments and offers optional extras for multimodal models, distributed reward‑model inference, and more.

  • Dispatch
    Hugging Face Skills: Enabling AI Agents to Fine-Tune LLMs

    Hugging Face has introduced Hugging Face Skills, a tool that allows coding agents like Claude Code to manage the entire LLM fine-tuning lifecycle, from hardware selection to model deployment.

  • Project
    OpenDCAI/DataFlow

    DataFlow is an open‑source Python framework that lets you build, run, and share low‑code pipelines for generating, cleaning, and evaluating LLM training data. It provides a library of reusable operators, a visual WebUI, an AI‑assistant for auto‑creating pipelines, and a Ray‑based distributed execution layer.

  • Project
    lofcz/LLMTornado

    A provider-agnostic .NET SDK for building AI agents and workflows with built-in connectors to 30+ API providers and vector databases.

  • Project
    EvolvingLMMs-Lab/lmms-eval

    A unified evaluation toolkit for Large Multimodal Models that provides reproducible and efficient benchmarking across image, video, and audio tasks.