allenai/OLMo-core

PyTorch building blocks for the OLMo ecosystem

What it solves

OLMo-core provides the foundational building blocks and training infrastructure needed to develop, train, and run inference on the OLMo family of large language models. It simplifies the process of training high-performance models by providing official scripts and integration with optimized hardware kernels.

How it works

The library serves as a core modeling and training framework. It integrates with several high-performance backends to optimize memory and speed, including FlashAttention, TransformerEngine, Liger-Kernel for fused-linear loss, and torchao for float8 training. It also supports Mixture-of-Experts (MoE) models via grouped_gemm. Users can launch training using official scripts via torchrun or the Beaker CLI, and perform inference through Hugging Face Transformers, vLLM, or the library's own built-in generation tools.

Who it’s for

This project is designed for AI researchers and engineers who are training large-scale language models from scratch or continuing the training (annealing) of existing OLMo checkpoints.

Highlights

  • Official Training Scripts: Includes ready-to-use scripts for OLMo-2 and OLMo-3 model families.
  • Hardware Optimization: Supports float8 training and low-memory fused-linear loss.
  • Flexible Inference: Compatible with industry-standard tools like vLLM and Hugging Face Transformers.
  • MoE Support: Includes capabilities for dropless mixture-of-experts models.

Related

  • Project
    PaddlePaddle/PaddleFormers

    A high-performance Transformers library for PaddlePaddle that enables efficient pre-training and post-training of LLMs and VLMs with extensive support for distributed strategies and domestic AI hardware.

  • Project
    baidu-baige/LoongForge

    A high-performance training framework for LLMs, VLMs, diffusion, and embodied models that optimizes parallelism and kernel efficiency to reduce training costs.

  • Project
    scaleapi/llm-engine

    An open-source engine for fine-tuning and serving large language models, providing tools to deploy models on hosted or self-managed Kubernetes infrastructure.

  • Project
    vllm-project/speculators

    A library for training speculative decoding draft models that integrate directly with vLLM to reduce LLM inference latency without losing output quality.

  • Project
    EvolvingLMMs-Lab/lmms-engine

    A unified, high-performance training engine for multimodal models that integrates distributed training, kernel fusion, and advanced optimizers to enable scalable pretraining and fine-tuning.