microsoft/antares

Antares: an automatic engine for multi-platform kernel generation and optimization. Supporting CPU, CUDA, ROCm, DirectX12, GraphCore, SYCL for CPU/GPU, OpenCL for AMD/NVIDIA, Android CPU/GPU backends.

What it solves

AutoRT (the next generation of Antares) is a compiler solution designed to help developers optimize and benchmark performance on various hardware accelerators. It specifically addresses the difficulty of creating high-performance operators for PyTorch 2.0, allowing users to bridge the gap between high-level PyTorch code and specific hardware backends like Windows DirectX 12 and Linux CUDA.

How it works

AutoRT acts as a backend for PyTorch 2.0. It uses an intermediate representation (IR) to define operators. Users can define custom, fused operators using either a Python API or a command-line interface, which AutoRT then compiles and optimizes for the target device. It can be used as a benchmark utility for profiling device performance or to accelerate standard PyTorch applications by generating a device-specific runtime.

Who it’s for

Hardware accelerator developers and PyTorch users who need to perform device performance testing, profiling, and the creation of custom high-performance operators for non-standard or new hardware backends.

Highlights

  • Multi-Backend Support: Experimental support for Windows DirectX 12 and Linux CUDA, with goals to support ROCm, OpenCL, SYCL, and Apple Metal.
  • Custom Operator Generation: Ability to create custom or fused operators that go beyond built-in PyTorch functions using a simple IR.
  • PyTorch 2.0 Integration: Seamlessly integrates as a backend for PyTorch 2.0 to accelerate applications like MNIST training and LLM inference (e.g., Llama 2).
  • Benchmarking Utility: Built-in tools for profiling and testing the performance of the target device.

Related

  • Project
    pytorch/TensorRT

    Torch-TensorRT integrates NVIDIA TensorRT into PyTorch to accelerate inference latency by up to 5x on NVIDIA GPUs.

  • Project
    huggingface/accelerate

    A PyTorch wrapper that abstracts the boilerplate code for distributed training, allowing users to run the same script on any device configuration including multi-GPU and TPU.

  • Project
    pytorch/benchmark

    PyTorch Benchmarks is a collection of standardized, lightweight versions of popular deep‑learning models (BERT, ResNet, Stable Diffusion, etc.) that can be run to measure the performance of different PyTorch builds, CUDA versions, and back‑ends. It offers a uniform API, install scripts, and multiple runners (simple test, pytest‑benchmark, custom userbenchmark) plus utilities for low‑noise tuning on AWS g4dn.metal instances.

  • Project
    pypose/bae

    A PyTorch-based library for exact 2nd-order sparse optimization in robotics, specifically designed to accelerate Bundle Adjustment and Pose Graph Optimization using CUDA.

  • Project
    MiroPsota/torch_packages_builder

    A toolset for automating the build and indexing of PyTorch-based packages with custom operators, providing pre-built wheels via a PEP 503 compliant index.