Xilinx/XRT

Run Time for AIE and FPGA based platforms

What it solves

XRT provides a standardized, abstracted software interface to manage and interact with AMD NPUs and FPGAs. This removes the need for developers to write device-specific code for different hardware platforms, allowing software to run across various AMD accelerator hardware like Alveo cards and Versal Adaptive SoCs.

How it works

It operates as a combination of user-space components and kernel drivers. On Linux, it utilizes the accel driver model, and on Windows, it uses the MCDM driver model. It supports both x86_64 and aarch64 CPU architectures and includes a command-line tool called xrt-smi for device configuration and validation.

Who it’s for

Developers working with AMD hardware accelerators, specifically those deploying workloads to NPUs and FPGAs.

Highlights

  • Supports a wide range of AMD hardware including Ryzen, Versal, Alveo, and Zynq UltraScale+ MPSoCs.
  • Cross-platform compatibility with Linux and Windows.
  • Includes xrt-smi for device management and validation.
  • Provides a unified API for seamless access to accelerator hardware.

Related

  • Project
    amd/xdna-driver

    A Linux kernel driver and XRT SHIM library that enables AI inference acceleration on AMD XDNA (NPU) devices for RyzenAI processors.

  • Project
    pykeio/ort

    A Rust interface for hardware-accelerated inference and training of ONNX machine learning models, enabling efficient deployment across various hardware.

  • Project
    google-ai-edge/LiteRT

    LiteRT is Google’s high‑performance on‑device runtime for running ML and generative‑AI models across phones, laptops, browsers, and IoT devices. It supersedes TensorFlow Lite with a unified Compiled Model API that automatically selects the best accelerator (CPU, GPU, NPU, WebGPU). The ecosystem includes converters for PyTorch, a CLI for coding‑agent workflows, a Web‑focused LiteRT.js, and a library for on‑device LLM inference. Supported platforms span Android, iOS, Linux, macOS, Windows, Web, and Raspberry Pi, with broad hardware acceleration support.

  • Project
    vllm-project/vllm-ascend

    vLLM Ascend is a hardware plugin that enables the vLLM LLM serving engine to run on Huawei Ascend NPUs. It supports a range of Atlas NPU hardware, requires Linux, Python 3.10‑3.12, CANN 9.1.0, PyTorch 2.10.0 + TorchNPU, and matches specific vLLM versions. The project provides regular releases, detailed docs, and an active community (Slack, forum, weekly meetings). It’s licensed under Apache 2.0.

  • Project
    intel/compute-runtime

    An open-source driver project that enables oneAPI Level Zero and OpenCL compute support for Intel graphics hardware architectures.