pnnx/pnnx

pnnx – PyTorch Neural Network eXchange

What it is

An open‑source tool that lets you take a PyTorch model, optimise its computation graph, and export it to a portable format that can be run with the lightweight ncnn inference engine or as a reduced‑dependency PyTorch script.
It defines a standard “PNNX” model representation that mirrors PyTorch operators, enabling loss‑less conversion between torchscript, ONNX, tnn and the native PNNX format.

Why you might care

Speed & size – The optimiser can prune unused ops and fuse layers, producing a smaller, faster model for edge devices.
Fewer dependencies – The generated PNNX model can be run without a full PyTorch installation (or even CUDA), which is handy for mobile, embedded, or server‑less deployments.
Interoperability – Convert models to ONNX‑zero or ncnn formats with a single command, making it easy to move between frameworks.

Installation

Method	Platform	How to install
Python package (recommended)	Windows / Linux / macOS (64‑bit)	`pip3 install pnnx`
Portable binaries	Same platforms (no Python needed)	Download the zip from the latest GitHub release and unzip. The binary works out‑of‑the‑box – no CUDA or PyTorch runtime required.
Build from source	Any platform with a C++ toolchain	Clone the `Tencent/ncnn` repo, then build the `tools/pnnx` folder with CMake, pointing it at your libtorch/libtorchvision installations.

Typical workflow

Export your PyTorch model to a torchscript (.pt) or ONNX (.onnx) file.
Run pnnx on that file, supplying the input shape(s). Example:
```
pnnx resnet18.pt inputshape=[1,3,224,224]
```
This produces several artefacts:
- resnet18.pnnx.param & resnet18.pnnx.bin – the PNNX graph and weights.
- resnet18_pnnx.py – a pure‑Python script that reconstructs the model without needing the original source code.
- resnet18.ncnn.param & resnet18.ncnn.bin – ready‑to‑run files for the ncnn engine.
- Optional ONNX‑zero file (resnet18.pnnx.onnx).

Run inference using the generated Python script or the ncnn runtime:

import torch, pnnx
model = pnnx.load('resnet18.pnnx.param', 'resnet18.pnnx.bin')
out = model(torch.rand(1,3,224,224))

python resnet18_ncnn.py

Validate that the outputs match the original PyTorch model (they should be numerically close).

Key options (passed on the command line)

fp16=1 – store weights in half‑precision, reducing file size.
optlevel=0|1|2 – control how aggressively the graph is optimised (0 = none, 2 = strongest).
device=cpu|gpu – choose the device for tracing torchscript models.
customop= – path(s) to compiled Torch extensions for custom operators.
moduleop= – list of modules to keep as a single fused operator.

When to use it

Deploying PyTorch models to mobile or embedded platforms where installing the full PyTorch stack is impractical.
Wanting a single, version‑stable model file that can be visualised in tools like Netron.
Converting models from other frameworks (via ONNX) into a format that ncnn can execute efficiently.

Resources

GitHub repo: https://github.com/pnnx/pnnx
Binary releases: https://github.com/pnnx/pnnx/releases
Visualization: drag the generated .pnnx.param or .pnnx.onnx into https://netron.app/ to explore the graph.

All details above are taken directly from the project's README.