microsoft/antares

Antares: an automatic engine for multi-platform kernel generation and optimization. Supporting CPU, CUDA, ROCm, DirectX12, GraphCore, SYCL for CPU/GPU, OpenCL for AMD/NVIDIA, Android CPU/GPU backends.

What it solves

AutoRT is a compiler solution designed to help runtime users create, benchmark, and optimize operators for PyTorch using their own hardware accelerators. It simplifies the process of writing custom kernels for different backends like CUDA and ROCm.

How it works

AutoRT functions as a compiler that allows users to define functions and bindings for hardware accelerators. Users can write kernel bodies using CUDA/HIP C++ 17 standards, following a specific syntax for defining inputs and outputs (e.g., @DEF_FUNC and @DEF_BIND).

Who it’s for

Developers and runtime users who need to optimize PyTorch operators and target specific hardware accelerators.

Highlights

  • Multi-backend support: Supports CUDA and ROCm environments.
  • C++ 17 Standard: Kernel bodies follow the CUDA/HIP C++ 17 standard.
  • C++ 17 Standard: Kernel bodies kernels can be defined and CLI-driven via the autort command.
  • Agentic workflow: Includes a tutorial that suggests an agentic approach to learning and implementing kernels.

Related

  • Project
    pytorch/TensorRT

    Torch‑TensorRT is an open‑source library that lets you compile any PyTorch model to run with TensorRT speed on NVIDIA GPUs. Use a single line (`torch.compile(..., backend="tensorrt")`) or an export workflow to produce a serialized module that can be loaded in Python or C++. It supports Linux, Windows, and Jetson, works with CUDA 13.4 + TensorRT 11.3, and is installable via pip or NVIDIA containers. Ideal for developers who want fast inference for vision, diffusion, or LLM models without leaving the PyTorch ecosystem.

  • Project
    huggingface/accelerate

    Accelerate is a thin PyTorch wrapper that removes the boilerplate for running training scripts on multiple GPUs, TPUs, or with mixed precision. You keep your own training loop and add just a few lines (an Accelerator object, a prepare() call, and accelerator.backward()). It also offers an optional CLI (accelerate config / accelerate launch) and a notebook launcher. It supports CPU, single/multi-GPU, multi-CPU, TPU, FP16/BF16/FP8 mixed precision, plus experimental DeepSpeed, FSDP, and Megatron-LM. Use it if you want control over your training loop without distributed-setup hassle; skip it if you want a fully high-level framework. Requires Python 3.11+ and PyTorch 2.6.0+; install via pip install accelerate.

  • Project
    pytorch/benchmark

    PyTorch Benchmarks is a collection of standardized, lightweight versions of popular deep‑learning models (BERT, ResNet, Stable Diffusion, etc.) that can be run to measure the performance of different PyTorch builds, CUDA versions, and back‑ends. It offers a uniform API, install scripts, and multiple runners (simple test, pytest‑benchmark, custom userbenchmark) plus utilities for low‑noise tuning on AWS g4dn.metal instances.

  • Project
    pypose/bae

    bae is a PyTorch library providing efficient, exact second-order sparse optimization algorithms for robotics, including Bundle Adjustment and Pose Graph Optimization. It leverages CUDA acceleration and integrates with PyTorch's autograd for high-performance non-linear least squares problems.

  • Project
    MiroPsota/torch_packages_builder

    A toolset for automating the build and indexing of PyTorch-based packages with custom operators, providing pre-built wheels via a PEP 503 compliant index.