MakazhanAlpamys/Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

What it solves

Soup simplifies the complex process of fine-tuning and post-training Large Language Models (LLMs). It eliminates the need for manual infrastructure management, SSH access to GPU servers, and complex configuration files, allowing users to start training with a single command.

How it works

Soup provides a command-line interface (CLI) and a Web UI that abstracts the training stack (PyTorch, Transformers, PEFT, TRL). It uses a single YAML configuration file to define the base model, task (e.g., SFT, DPO, GRPO), and training parameters.

Key technical features include:

  • Layer Streaming: A beta feature that allows fine-tuning large models (like an 8B model) on consumer GPUs with as little as 4 GB of VRAM by feeding decoder layers to the GPU one at a time.
  • Multiple Backends: Support for standard transformers, Unsloth (for faster training), and MLX (for Apple Silicon).
  • Comprehensive Tooling: Integrated commands for data inspection, batch inference, model merging, and exporting to formats like GGUF for Ollama.

Who it’s for

Developers and ML engineers who want to fine-tune LLMs without fighting infrastructure, as well as those with limited hardware resources (e.g., laptop GPUs) who want to leverage techniques like QLoRA and layer streaming.

Highlights

  • One-Command Workflow: Initialize with soup init and train with soup train.
  • Low VRAM Requirements: Capability to train 8B models on 4 GB VRAM via layer streaming.
  • Broad Task Support: Supports SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO, BCO, and more.
  • Extensive Export Options: Export models to GGUF, ONNX, TensorRT, AWQ, and GPTQ.
  • Integrated Web UI: Local dashboard for experiment tracking, dataset exploration, and model chat.

Related

  • Dispatch
    Soup Enables Fine‑Tuning an 8B Model on a 4 GB Laptop GPU via Layer Streaming

    Soup lets you fine‑tune an 8B parameter LLM on a 4 GB laptop GPU using layer streaming and QLoRA, achieving bit‑exact results with minimal overhead.

  • Project
    hiyouga/LlamaFactory

    LlamaFactory is a unified, zero-code framework that enables users to fine-tune, evaluate, and deploy over 100 large language models through a simple Web UI or command-line interface.

  • Project
    lyogavin/airllm

    AirLLM is a Python library that streams LLM layers from disk so only one layer (plus optional adapters) resides on the GPU at a time, enabling inference of models up to 671 B parameters on as little as 4‑12 GB VRAM and fine‑tuning of 125 B models on ~6 GB. It offers optional 4‑/8‑bit weight‑only compression for up to 3× speed‑up and works with most Hugging Face models via a familiar `AutoModel` API.

  • Project
    FujitsuResearch/OneCompression

    OneComp is a Python library that automatically compresses large language models using advanced post‑training quantisation (QEP, LPCD, AutoBit, MDBF, etc.), picks per‑layer bit‑widths based on available VRAM, and provides tools for fine‑tuning, export (GGUF, OpenVINO), and serving (vLLM). It works via a one‑line CLI or a simple Python API.

  • Project
    huggingface/open-r1

    An open-source reproduction of the DeepSeek-R1 pipeline, providing tools for SFT, GRPO reinforcement learning, and synthetic data generation to create reasoning models.