tenstorrent/tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

What is tt‑metal?

tt‑metal (also called TT‑Metalium) is an open‑source software stack that lets developers write and run neural‑network kernels on Tenstorrent’s custom AI accelerators (the Wormhole and Blackhole chips). It provides two layers:

  • TT‑Metalium – a low‑level C++/Python programming model for building kernels that run directly on the hardware. It includes a programming guide, a set of tech‑reports on memory layout, data formats, matrix engines, and tools for debugging and profiling.
  • TT‑NN – a higher‑level Python library that implements common neural‑network operations (matmul, convolutions, attention, etc.) on top of TT‑Metalium, making it easier to run full models.

The repo ships with:

  • Example kernels (hello‑world, add integers, matmul, eltwise, DRAM movement).
  • A collection of model demos (Llama 3.3 70B, Qwen 2.5 7B/72B, Whisper, Mixtral 8x7B) that showcase inference performance on Tenstorrent hardware.
  • Technical reports that document performance optimizations, memory allocators, tensor layouts, flash‑attention, and multi‑chip scaling.
  • Tooling such as a visualizer, low‑level debugger (TT‑Exalens), system‑management CLI (TT‑SMI), and profiling utilities (Tracy, Watcher, Inspector).

Who is it for?

  • Hardware engineers who need to write custom kernels for Tenstorrent devices.
  • ML researchers/engineers who want to run large language models or vision models on Tenstorrent accelerators with high throughput.
  • Tool developers building debugging, profiling, or deployment pipelines for this hardware.

How to get started

  1. Install the stack following the INSTALLING.md guide (pre‑built many‑linux wheels are provided).
  2. Run the simple “hello‑world” or “add integers” examples to verify kernel execution.
  3. Explore the model demos under models/demos/ to see end‑to‑end inference (e.g., Llama 3.3 70B on a 32‑device Galaxy mesh).
  4. Dive into the programming guide and tech‑reports for deeper performance tuning.

Why it matters

Tenstorrent’s chips are built for high‑throughput tensor operations with a unique architecture (matrix engine, SFPU, Ethernet‑based mesh). tt‑metal abstracts the hardware details while still exposing enough control for expert kernel developers, enabling the community to push the limits of LLM and vision model performance on this emerging AI accelerator.

All details above are taken directly from the repository’s README and linked documentation.