modelscope/ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

What it solves

ms-swift provides a comprehensive, scalable infrastructure for the entire lifecycle of large language models (LLMs) and multimodal models. It simplifies the complex process of pre-training, fine-tuning, human alignment, evaluation, quantization, and deployment, allowing users to move from raw data to a deployed model using a single framework.

How it works

The framework integrates various training technologies and acceleration engines to optimize performance across different hardware. It supports a vast array of models (600+ text-only and 400+ multimodal) and provides multiple training paths:

  • Lightweight Fine-Tuning: Implements methods like LoRA, QLoRA, and DoRA to reduce resource requirements.
  • Distributed Training: Utilizes DeepSpeed, FSDP, and Megatron parallelism (TP, PP, CP, EP) to handle massive models and MoE architectures.
  • Reinforcement Learning: Includes a wide family of GRPO algorithms (GRPO, DAPO, GSPO, etc.) for human alignment.
  • Optimization: Employs memory-saving techniques like GaLore, UnSloth, and Flash-Attention 2/3.
  • Deployment: Integrates with vLLM, SGLang, and LMDeploy for high-performance inference.

Who it’s for

It is designed for AI researchers and developers who need to fine-tune and deploy state-of-the-art large models efficiently, especially those working with multimodal data or those requiring high-scale distributed training on diverse hardware (NVIDIA, AMD, Ascend, etc.).

Highlights

  • Massive Model Support: Day-0 support for over 1,000 combined text and multimodal models.
  • Hardware Agnostic: Works across NVIDIA GPUs, AMD GPUs, Apple MPS, CPUs, and domestic NPUs/GPUs.
  • Full-Pipeline: Covers everything from dataset preparation and training to quantization and OpenAI-compatible deployment.
  • Advanced RL: Built-in support for the GRPO algorithm family and other preference learning methods like DPO and ORPO.
  • User-Friendly: Offers both a command-line interface and a zero-threshold Web-UI for non-experts.

Related

  • Project
    NVIDIA-NeMo/RL

    NeMo RL is an open-source post-training library for scalable reinforcement learning of large multimodal models (LLMs, VLMs). It provides algorithms like GRPO, DAPO, and SFT, with flexible backends for distributed training and generation.

  • Project
    Lightning-AI/litgpt

    LitGPT is a high-performance, abstraction-free toolkit for pretraining, finetuning, and deploying over 20 popular LLMs at enterprise scale.

  • Project
    OptimalScale/LMFlow

    LMFlow is an extensible Python toolbox for fine‑tuning large language models. It supports full‑parameter training, LoRA, the memory‑efficient LISA algorithm, QLoRA quantization, and a suite of speed‑up tricks (FlashAttention‑2, DeepSpeed, vLLM, SGLang). After training, it provides simple scripts for chat‑style inference and optional Gradio/Flask deployment. The library is designed for Linux/Colab environments and offers optional extras for multimodal models, distributed reward‑model inference, and more.

  • Project
    2U1/Qwen-VL-Series-Finetune

    A training toolkit for fine-tuning Qwen-VL series models (Qwen2-VL, Qwen2.5-VL, Qwen3-VL, Qwen3.5) supporting SFT, DPO, GRPO, and multimodal data.

  • Project
    huggingface/trl

    A comprehensive library for post-training foundation models using techniques like SFT, DPO, and GRPO to align them with human preferences.