ServiceNow/Fast-LLM

Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research

What it solves

Fast-LLM is designed to accelerate the training of large language models (LLMs), reducing both the time and cost associated with training models ranging from 1B to 70B+ parameters. It addresses the challenges of scalability and memory efficiency when training massive models across large GPU clusters.

How it works

Built on PyTorch and Triton, the library uses fine-tuned kernels and advanced parallelism techniques to maximize throughput. It implements 3D parallelism (Data, Tensor, and Pipeline parallelism), sequence length parallelism, and ZeRO-1, 2, and 3 for memory optimization. It also supports mixed precision training and gradient accumulation to handle large batches.

Who it’s for

AI research teams and practitioners who need to train large-scale generative AI models from scratch or fine-tune them using high-performance computing clusters (such as Slurm or Kubernetes).

Highlights

  • High Throughput: Achieves significant speedups, such as 9,800 tokens/s/H100 for Mistral-7B training.
  • Scalability: Supports distributed training across multiple nodes and GPUs with comprehensive parallelism options.
  • Flexibility: Compatible with common GPT-like architectures and includes an efficient dropless Mixture-of-Experts (MoE) implementation.
  • Ease of Use: Provides pre-built Docker images, YAML-based configuration, and seamless integration with Hugging Face Transformers.

Related

  • Project
    linkedin/Liger-Kernel

    A collection of efficient Triton kernels for LLM training that increases throughput by 20% and reduces memory usage by 60% through kernel fusion and chunking.

  • Dispatch
    Training Language Models with Megatron-LM

    Hugging Face provides a guide on using NVIDIA's Megatron-LM framework to efficiently pre-train large language models on GPUs, including integration with the Transformers library.

  • Project
    NVIDIA/Megatron-LM

    A GPU-optimized library and reference framework for training massive transformer models at scale, supporting advanced parallelism strategies and mixed-precision training.

  • Project
    unslothai/unsloth-zoo

    A utility library for Unsloth that enables faster and more memory-efficient fine-tuning of LLMs, vision, and TTS models with significantly reduced VRAM requirements.

  • Project
    deepspeedai/DeepSpeed

    A deep learning optimization library that enables the training of models with trillions of parameters by optimizing memory and compute efficiency across distributed systems.