Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

What it solves

LitGPT provides a high-performance, enterprise-ready toolkit for working with Large Language Models (LLMs). It removes complex abstraction layers, allowing developers to pretrain, finetune, evaluate, and deploy over 20 different LLMs with full control and minimal overhead.

How it works

LitGPT implements models from scratch without internal abstraction layers, which makes the code easier to debug and highly performant. It uses a command-line interface (CLI) to manage workflows, such as downloading model weights, running interactive chats, or executing training jobs on your own data. The project is designed to scale from a single GPU to large clusters, utilizing optimized techniques like quantization and memory-efficient training methods.

Who it’s for

It is designed for developers and researchers who need a transparent, high-performance environment to train or deploy LLMs. Because it avoids complex abstractions and provides single-file implementations, it is particularly useful for those who want to understand or customize the underlying model mechanics for production-scale applications.

Highlights

  • Supports 20+ popular LLMs including Llama 3, Gemma 2, Qwen 2.5, and Phi 4.
  • Provides optimized, enterprise-scale recipes for pretraining and finetuning.
  • Includes built-in support for quantization (fp4/8/16/32) to run models on low-memory hardware.
  • Offers a simple CLI for common tasks like model evaluation, interactive chat, and API deployment.
  • Apache 2.0 licensed, making it suitable for unlimited enterprise use.

Related

  • Project
    alibaba/rtp-llm

    A high-performance LLM inference acceleration engine developed by Alibaba that optimizes throughput and latency for production-scale deployments.

  • Project
    google-ai-edge/LiteRT-LM

    A production-ready orchestration layer for running high-performance, cross-platform LLMs on-device using hardware acceleration.

  • Project
    modelscope/ms-swift

    A scalable infrastructure for fine-tuning and deploying 1,000+ text and multimodal large models, supporting the full pipeline from training to quantization and inference.

  • Project
    mlc-ai/web-llm

    A high-performance in-browser LLM inference engine that uses WebGPU to run language models locally without server-side processing.

  • Project
    opendilab/LightRFT

    LightRFT is an open‑source, distributed reinforcement‑learning framework for fine‑tuning large language and multimodal generative models. It unifies training back‑ends (DeepSpeed, FSDP) with rollout engines (SGLang, vLLM), supports a range of RL algorithms (GRPO, REINFORCE++, DAPO, etc.), and works across text, image, video, and audio modalities. The library provides ready‑to‑run examples (e.g., GSM8K with Qwen2.5), extensive docs, and Docker images, making it suitable for researchers and engineers building RL‑based alignment pipelines.