Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

What it solves

LitGPT provides a high-performance, minimal framework for working with over 20 popular Large Language Models (LLMs). It eliminates complex abstraction layers, giving developers full control over the model implementations to maximize performance, reduce costs, and simplify debugging at enterprise scale.

How it works

Every supported LLM is implemented from scratch in a single file to remove internal abstractions. The project provides a unified command-line interface (CLI) and Python API to handle the entire LLM lifecycle: pretraining, continued pretraining, finetuning, evaluation, and deployment. It leverages advanced optimization techniques like Flash Attention, FSDP, and quantization (fp4/8/16/32) to reduce GPU memory usage and scale from 1 to 1,000+ GPUs/TPUs.

Who it’s for

It is designed for developers and enterprise teams who need a performant, transparent LLM framework that supports a wide range of models (such as Llama 3, Gemma 2, Phi 4, and Qwen 2.5) and requires precise control over training and inference.

Highlights

  • Broad Model Support: Supports 20+ high-performance LLMs including Llama, Gemma, Phi, and Qwen.
  • No Abstractions: Single-file implementations for easier debugging and maximum performance.
  • Full Lifecycle Management: Integrated tools for pretraining, finetuning (LoRA, QLoRA, Adapter), and deployment.
  • Enterprise Scaling: Optimized for scale with support for FSDP and massive GPU/TPU clusters.
  • Fast Deployment: Built-in serve command to quickly turn a model into an inference API.

Related

  • Project
    Lightning-AI/pytorch-lightning

    A deep learning framework that abstracts away the engineering boilerplate of PyTorch, enabling researchers to pretrain and finetune AI models at scale across various hardware.

  • Project
    modelscope/ms-swift

    A scalable infrastructure for the fine-tuning, evaluation, and deployment of over 1,000 text and multimodal large models.

  • Project
    opendilab/LightRFT

    LightRFT is an open‑source, distributed reinforcement‑learning framework for fine‑tuning large language and multimodal generative models. It unifies training back‑ends (DeepSpeed, FSDP) with rollout engines (SGLang, vLLM), supports a range of RL algorithms (GRPO, REINFORCE++, DAPO, etc.), and works across text, image, video, and audio modalities. The library provides ready‑to‑run examples (e.g., GSM8K with Qwen2.5), extensive docs, and Docker images, making it suitable for researchers and engineers building RL‑based alignment pipelines.

  • Project
    AI4Finance-Foundation/FinGPT

    An open-source framework for financial large language models that enables cost-effective fine-tuning and adaptation of open-source LLMs for specialized financial tasks.

  • Project
    alibaba/rtp-llm

    A high-performance LLM inference acceleration engine developed by Alibaba that optimizes throughput and latency for production-scale deployments.