amd/RyzenAI-SW

AMD Ryzen™ AI Software includes the tools and runtime libraries for optimizing and deploying AI inference on AMD Ryzen™ AI powered PCs.

What it solves

It provides the tools and runtime libraries needed to optimize and deploy AI inference on AMD Ryzen AI-based PCs, allowing developers to leverage hardware acceleration for AI applications.

How it works

The software acts as a bridge between AI models and the Ryzen AI hardware, offering a set of runtime libraries and tools that enable efficient execution of various models. It supports integration with the ONNX Runtime and Windows ML, and provides specialized examples for different model types including CNNs, Transformers, and LLMs.

Who it’s for

Developers building AI applications specifically for AMD Ryzen AI-powered hardware who want to maximize performance and efficiency.

Highlights

  • Support for Large Language Models (LLMs) using the ONNX Runtime GenAI API, including RAG and Vision Language Models (VLM).
  • Capabilities for image classification, object detection (YOLOv8), and super-resolution.
  • Integration with Windows ML for running Transformers and LLMs.
  • Support for Automatic Speech Recognition (ASR) using OpenAI Whisper and NVIDIA Parakeet TDT.
  • Tools for model quantization via AMD Quark.

Related

  • Project
    jamjamjon/usls

    A cross-platform Rust library powered by ONNX Runtime for efficient inference of vision and vision-language models under 1B parameters.

  • Project
    ROCm/FastFlowLM

    A lightweight, NPU-first runtime for running LLMs, VLMs, and other AI models on AMD Ryzen™ AI NPUs with high power efficiency and no GPU required.

  • Project
    NVIDIA/TensorRT-LLM

    An open-source library from NVIDIA that optimizes inference for LLMs and visual generation models using specialized kernels and an efficient runtime to maximize throughput and latency on NVIDIA GPUs.

  • Project
    qualcomm/ai-hub-apps

    A collection of sample apps and tutorials for deploying machine learning models on Qualcomm devices, supporting various runtimes and Snapdragon NPU acceleration.

  • Project
    microsoft/onnxruntime

    A cross-platform machine learning accelerator that optimizes inference and training for models from various frameworks like PyTorch and TensorFlow.