Tencent/ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform

What it solves

ncnn provides a high-performance framework for running (inferencing) deep learning models on resource-constrained environments. It enables developers to take models trained in frameworks like PyTorch and deploy them efficiently on mobile phones, embedded systems, desktop computers, and web browsers without requiring heavy third-party runtime dependencies.

How it works

The framework uses a two-part approach: a conversion tool (pnnx) and a lightweight inference engine. Developers first convert their PyTorch or ONNX models into the ncnn format using pnnx. Once converted, the ncnn engine—which supports both CPU and Vulkan GPU acceleration—loads these models to perform inference. It is designed to be self-contained, meaning it does not rely on external runtime libraries, making it highly portable across various hardware architectures.

Who it’s for

This project is for software engineers and machine learning practitioners who need to deploy AI models on edge devices, mobile apps, or desktop applications where performance and minimal dependencies are critical. It is widely used in production environments, including major applications like WeChat and QQ.

Highlights

  • High-performance inference optimized for mobile, embedded, and desktop platforms.
  • Zero third-party runtime dependencies for easier integration.
  • Dual-backend support utilizing both CPU and Vulkan GPU acceleration.
  • Includes pnnx for seamless conversion from PyTorch and ONNX models.
  • Broad hardware support, including Android, iOS, Windows, Linux, macOS, WebAssembly, and various edge computing platforms.

Related

  • Project
    alibaba/MNN

    A lightweight deep learning framework optimized for high-performance inference and training on mobile, IoT, and embedded devices.

  • Project
    mil-tokyo/webdnn

    A browser-based neural network inference engine that runs ONNX models directly in the web browser using WebGPU, WebGL, and WebAssembly.

  • Project
    nndeploy/nndeploy

    A high-performance AI deployment framework that uses visual workflows to deploy AI models across desktop, mobile, and edge devices.

  • Project
    zuoqing1988/ZQCNN

    A cross-platform inference framework for Windows, Linux, and ARM-Linux that provides optimized CPU kernels for efficient model execution, featuring specialized demos for face detection and recognition.

  • Project
    pnnx/pnnx

    pnnx is an open‑source tool that optimises PyTorch models and exports them to a lightweight, dependency‑free format (PNNX) and to ncnn/ONNX‑zero files, enabling fast inference on edge devices.