ARM-software/armnn

What it solves

Arm NN is a machine learning inference engine designed to accelerate ML models on Arm Cortex-A CPUs and Arm Mali GPUs. It bridges the gap between high-level neural network frameworks and power-efficient Arm hardware, ensuring that models run with higher performance and power efficiency than generic ML libraries.

How it works

It utilizes the Arm Compute Library (ACL) to apply architecture-specific optimizations (such as SVE2) to the hardware. It supports models in TensorFlow Lite (TF Lite) and ONNX formats. The engine can be integrated via a TF Lite Delegate (which provides the widest operator support by falling back to the reference TF Lite runtime for unsupported operators) or through dedicated TF Lite and ONNX parsers.

Who it’s for

Developers building ML applications for Android and Linux platforms targeting Arm-based hardware, including those using Python or C++ APIs.

Highlights

• Hardware Acceleration: Specifically optimized for Arm Cortex-A CPUs, Arm Mali GPUs, and Arm Ethos-N NPUs.
• Flexible Integration: Offers a TF Lite Delegate for easy acceleration and C++ parsers for direct integration.
• Cross-Platform Support: Provides pre-built binaries for various Android API levels and Linux Arm v8a/v8.2a.
• Customizable Builds: Includes a build tool with Dockerfile support to allow users to select specific components for their project.