nndeploy/nndeploy

What it solves

nndeploy is designed to simplify and accelerate the deployment of AI algorithms on edge devices and local hardware. It addresses the challenges of porting models to diverse platforms (Windows, macOS, Android, iOS, NVIDIA Jetson, Ascend, RK) and optimizing performance for real-time inference.

How it works

The framework uses a visual workflow editor where users can drag and drop nodes to build an AI pipeline. These workflows are exported as JSON files, which can then be loaded and executed via C++ or Python APIs. To ensure high performance, it integrates 13 different inference engines (such as TensorRT, ONNXRuntime, and OpenVINO) and implements optimization strategies like pipeline parallelism, zero-copy memory management, and memory pools.

Who it’s for

It is intended for AI developers and engineers who need to deploy models across multiple platforms and cross-platform compatibility without rewriting code for each hardware target.

Highlights

• Visual Workflow: Drag-and-drop interface for building and debugging AI pipelines with real-time parameter adjustment.
• Multi-Platform Support: Unified deployment across desktop, mobile, and edge computing devices.
• Extensible: Supports custom nodes written in Python or C++/CUDA for specialized preprocessing or high-performance logic.
• Broad Engine Integration: Deep integration with 13 mainstream inference frameworks, allowing users to choose the best engine for their specific hardware.
• Out-of-the-box Models: Pre-deployed nodes for LLMs (QWen), image generation (Stable Diffusion), OCR, and object detection (YOLO series).