tenstorrent/tt-forge

Tenstorrent's MLIR Based Compiler. We aim to enable developers to run AI on all configurations of Tenstorrent hardware, through an open-source, general, and performant compiler.

What it solves

TT-Forge is an open-source AI compiler stack designed to make running AI workloads—including inference and training—on Tenstorrent hardware straightforward. It bridges the gap between high-level ML frameworks like PyTorch, JAX, and ONNX and the underlying Tenstorrent hardware, ensuring that if a model fits in memory, it can be executed.

How it works

TT-Forge operates as a multi-layered stack built on TT-Metalium. It uses various frontends to ingest models from different frameworks:

  • TT-XLA: The primary frontend for PyTorch and JAX, compiling models into StableHLO graphs.
  • TT-Forge-ONNX: A TVM-based frontend for ONNX, TensorFlow, and PaddlePaddle.
  • TT-MLIR: The core MLIR-based compiler that optimizes the graph (fusion, sharding, layout) and lowers it to TT-Metalium.
  • TT-Lang: A Python DSL that allows developers to write custom high-performance kernels in Python rather than C++.
  • TT-Blacksmith: Provides optimized training recipes and experiments for various model types.

Who it’s for

This project is for AI researchers and developers who want to deploy or train models on Tenstorrent hardware, as well as kernel developers who need to high-performance custom operations.

Highlights

  • Broad Framework Support: Supports PyTorch, JAX, ONNX, TensorFlow, and PaddlePaddle.
  • Extensive Model Library: Over 800 model variants tested in CI, including Llama 3, Stable Diffusion XL, and YOLOv12.
  • Singe and Multi-chip Support: Capable of running large models (like Llama 3 70B) across multiple chips (N300+).
  • Python-based Kernel Development: TT-Lang enables writing fused ops in Python with built-in simulation and profiling.

Related

  • Project
    tenstorrent/tt-metal

    tt‑metal (TT‑Metalium) is Tenstorrent’s open‑source stack for writing low‑level kernels (C++/Python) and high‑level neural‑network ops (TT‑NN) that run on Tenstorrent AI chips. It includes model demos, performance reports, and a suite of debugging/profiling tools, aimed at developers who want to run or optimise LLMs, vision models, and other AI workloads on Tenstorrent hardware.

  • Project
    openxla/xla

    XLA is an open-source machine learning compiler that optimizes models from PyTorch, TensorFlow, and JAX for high-performance execution on GPUs, CPUs, and ML accelerators.

  • Project
    microsoft/onnxruntime

    A cross-platform machine learning accelerator that optimizes inference and training for models from various frameworks like PyTorch and TensorFlow.

  • Project
    PINTO0309/onnx2tf

    A tool for converting ONNX models to LiteRT, TFLite, TensorFlow, and PyTorch, featuring a high-speed direct conversion path to minimize deployment friction.

  • Project
    tracel-ai/burn

    Burn is a unified tensor library and deep learning framework written in Rust that allows the same code to be used for both training and inference across diverse hardware.