pnnx/pnnx

What it solves

PNNX (PyTorch Neural Network eXchange) addresses the challenge of PyTorch model interoperability. It provides an open standard and model format that allows PyTorch models to be optimized and exported to various formats for efficient inference, reducing dependency on extension packages and simplifying the transition from training to deployment.

How it works

PNNX defines a computation graph and high-level operators that strictly match PyTorch. It can take PyTorch models (via TorchScript or ONNX) or other formats like TNN, and convert them into an optimized PNNX format. From there, it can export the model to portable PNNX files, ONNX-zero, or ncnn models for deployment.

Who it’s for

Developers and ML engineers who need to optimize PyTorch models for deployment on different hardware or inference engines, particularly those targeting the ncnn framework.

Highlights

• Multi-format support: Converts TorchScript, ONNX, and TNN files into optimized models.
• Graph Optimization: Offers multiple optimization levels to refine models specifically for inference.
• Portable Binaries: Provides standalone binary packages for Windows, Linux, and macOS that do not require a CUDA or PyTorch runtime environment.
• Flexible Export: Generates multiple output files including PNNX graph definitions, weights, and PyTorch/pyncnn inference scripts.