NVIDIA/TransformerEngine

NVIDIA Transformer Engine – Fast, low‑precision training for large Transformers

What it is – A C++/CUDA library (with Python bindings) that provides highly‑optimized building blocks for training Transformer‑style models on NVIDIA GPUs. It supplies fused kernels and automatic mixed‑precision‑style APIs for PyTorch and JAX, handling the bookkeeping needed for FP8, MXFP8, NVFP4, BF16 and FP16 training.

Why it matters – Modern LLMs and other large‑scale models are limited by the amount of compute and memory required for training. By moving most arithmetic to newer low‑precision formats (especially FP8 on Hopper/Blackwell/Rubin GPUs) and fusing multiple operations (attention, GEMM, normalization, MoE routing, etc.), Transformer Engine can dramatically increase throughput while preserving model quality. It also abstracts the low‑precision details so you can keep your existing training code.

Key capabilities

• FP8 / MXFP8 / NVFP4 training on NVIDIA Hopper, Ada, Blackwell and Rubin GPUs (the newest hardware that supports these formats).
• Optimized fused kernels for attention, linear layers, layer‑norm, MoE routing, and communication (tensor‑, sequence‑, context‑, and expert‑parallel).
• Automatic scaling‑factor management – the library tracks amax histories and quantization metadata, exposing an autocast‑style context that works like PyTorch’s native AMP.
• Framework support – thin, idiomatic APIs for both PyTorch (torch.nn.Module wrappers) and JAX (Flax‑compatible layers).
• Distributed‑training ready – integrates with NCCL, supports overlapping communication, zero‑copy expert‑parallel, and CUDA‑graph acceleration.
• Broad ecosystem integrations – works out‑of‑the‑box with Hugging Face Accelerate, PyTorch Lightning, NVIDIA Megatron‑LM, NVIDIA NeMo, Amazon SageMaker Model Parallel Library, and more.

Typical workflow

1. Install the package for your framework:

   pip install "transformer_engine[pytorch]"   # or [jax]
   

2. Wrap your model’s layers with the TE equivalents (e.g., te.Linear, te.LayerNorm, te.MultiHeadAttention).
3. Enclose the forward pass in an autocast context to enable FP8/FP16/BF16 automatically:

   with transformer_engine.autocast(dtype=torch.float8):
       output = model(input)
   

4. Run your usual training loop; TE handles scaling, quantization, and communication under the hood.

Getting started & resources

• Quick‑start docs – https://docs.nvidia.com/deeplearning/transformer-engine/getting_started/index.html
• API reference – PyTorch: https://docs.nvidia.com/deeplearning/transformer-engine/api/pytorch.html ; JAX: https://docs.nvidia.com/deeplearning/transformer-engine/api/jax.html
• Examples – a collection of ready‑to‑run notebooks and scripts in the repo’s examples/ folder.
• Deep‑dive guides – low‑precision training guide, FP8 primer, and the original FP8 paper (arXiv:2209.05433).
• Release notes – latest v2.19 adds Rubin GPU support, hybrid quantization, and expanded FP8 attention.

Who should use it

• Researchers and engineers training large language models, vision Transformers, or MoE models who have access to recent NVIDIA GPUs.
• Teams looking to cut training time and GPU memory usage without rewriting their PyTorch/JAX codebases.
• Anyone building on top of popular LLM stacks (Megatron‑LM, Hugging Face, NeMo) and wanting to leverage FP8 acceleration.

License – Apache 2.0 (open source).