stas00/ml-engineering

Machine Learning Engineering Open Book

What it is

An open, free collection of practical knowledge for training and fine-tuning large language models (LLMs) and multi-modal models, plus running their inference. It's written as an "open book" — a living document hosted on GitHub, with downloadable ebook versions (PDF and EPUB) also available.

Who it's for

Engineers and operators who train or run large AI models. The material is technical and hands-on, full of scripts and copy-paste commands meant to be used directly.

Author's background

The author, Stas Bekman, compiled this from real experience training the open-source BLOOM-176B model (2022), the IDEFICS-80B multi-modal model (2023), and RAG models at Contextual.AI (2024).

What's inside

The content is organized into seven parts:

  1. Insights — high-level guidance, such as how to choose a cloud provider and how to decide whether upgrading GPUs is worth the cost.
  2. Hardware — compute accelerators, CPUs, storage, and networking.
  3. Orchestration — managing containers and resources, including SLURM.
  4. Training — model training guides.
  5. Inference — model inference insights.
  6. Development — debugging and testing tips.
  7. Miscellaneous — resources and LLM/VLM chronicles.

Key tools included

Several ready-to-use scripts:

  • torch-dist-bench.py — an easier way to benchmark network throughput.
  • torch-distributed-gpu-test.py — quickly test inter-node connectivity.
  • mamf-finder.py — measure the actual computing speed (TFLOPS) you can get from an accelerator.

Extras

  • SKILL.md files — two files that teach AI agents how to do ML engineering and how to evaluate a GPU cluster.
  • A course — "Lessons Learned from Training LLMs," a different way to read the book by going through terse insights.
  • A recorded talk — a lecture on building resilient ML engineering skills.
  • Community discussions — a forum for sharing experience.
  • Comparison tables — for accelerator speed and memory, and network speeds.

Formats

Available as a GitHub repository and as downloadable PDF and EPUB ebook versions hosted on HuggingFace.

License

Distributed under Attribution-ShareAlike 4.0 International (CC-BY-SA).

Related works

Two companion open books by the same author: The Art of Debugging and Stas' Python Cookbook.

Related

  • Project
    huggingface/trl

    A comprehensive library for post-training foundation models using techniques like SFT, DPO, and GRPO to align them with human preferences.

  • Project
    fengdu78/lihang-code

    Python implementations of every algorithm from Li Hang's classic machine learning textbook, organized chapter by chapter.

  • Project
    Unity-Technologies/ml-agents

    A toolkit that turns Unity games and simulations into environments for training intelligent agents using reinforcement and imitation learning.

  • Project
    fengdu78/deeplearning_ai_books

    A comprehensive collection of Chinese-language study notes for Andrew Ng's Deep Learning Specialization, covering neural network fundamentals, optimization, and advanced architectures like CNNs and RNNs.