LeiWang1999/ZYNQ-NVDLA

ZYNQ‑NVDLA – NVDLA on Xilinx Zynq FPGA

What it is

• A collection of hardware and software artefacts that let you run NVIDIA‑DLA (Deep Learning Accelerator) neural‑network inference on a Xilinx Zynq‑7000 or Zynq MPSoC FPGA board.
• The project bundles the RTL for the small NVDLA configuration, a Vivado IP package, kernel‑mode drivers for PetaLinux, pre‑built ARM64 libraries, and a Tengine‑Lite backend that translates Tengine models into the OpenDLA format.

Key components

Folder	Content
IP/	Vivado IP package for the NVDLA‑small core (ready to be instantiated in a Zynq design).
RTL/	Verilog source of the NVDLA‑small RTL plus a thin wrapper for integration.
kmd/	Kernel‑mode driver for PetaLinux, with support for both Zynq‑7000 and Zynq MPSoC devices.
prebuilt/	AArch64‑compiled runtime libraries that the host CPU uses to talk to the accelerator.
include/	Header files needed by the Tengine backend.
umd/	Source of the compiler and runtime that turn a Tengine model into the OpenDLA binary format.
sdk_sanity/	Small sanity‑check programs that verify the accelerator works end‑to‑end.
reports/	Timing, power, resource‑utilisation and execution‑time reports from synthesis runs.
examples/ (built via Tengine‑Lite)	Ready‑made demo binaries for classification (ResNet‑18 on CIFAR‑10) and object detection (YOLOX‑nano).

How it works

1. Hardware – The NVDLA‑small IP is synthesized for a Zynq FPGA, producing a custom accelerator block that sits on the programmable logic (PL).
2. Driver – A Linux kernel driver (kmd) exposes the accelerator as a device node that the host CPU can control.
3. Software stack – Tengine‑Lite (an open‑source inference engine) is compiled with a special OpenDLA backend. Models are first converted to a .tmfile (Tengine’s binary format) and then to an OpenDLA binary that the NVDLA hardware can execute.
4. Runtime – The pre‑built ARM64 libraries in prebuilt/ handle memory management, command submission, and result retrieval.

Demo usage

• Classification – Run ResNet‑18 on a 32×32 CIFAR‑10 image:

  cd <tengine-lite-root>/build
  cmake --build . --target tm_classification_opendla
  ./examples/tm_classification_opendla -m models/resnet18-cifar10-nosoftmax-relu_int8.tmfile -i images/cat.jpg -g 32,32 -s 1,1,1
  

  The output shows the top‑5 class scores and reports ~12 ms inference time on the NVDLA block.

• Object detection – Run YOLOX‑nano on a picture of a dog:

  cmake --build . --target tm_yolox_opendla
  ./examples/tm_yolox_opendla -m models/yolox_nano_relu_int8.tmfile -i images/dog.jpg -r 1
  

  The demo prints three detections (car, dog, bicycle) with confidence percentages and bounding‑box coordinates.

Why it matters

• Edge AI – By moving inference from the CPU to a dedicated FPGA‑implemented DLA, latency drops dramatically while keeping power consumption low, which is ideal for embedded or edge devices.
• Open‑source stack – All hardware (RTL, IP), drivers, and software are openly available, allowing researchers and developers to experiment with custom accelerator configurations or integrate NVDLA into their own Zynq‑based products.

Getting started

1. Clone the repo and ensure you have Xilinx Vivado and PetaLinux installed.
2. Build the NVDLA IP (IP/) and integrate it into a Vivado block design for your Zynq board.
3. Compile the kernel driver (kmd/) and load it on the running PetaLinux image.
4. Install Tengine‑Lite, enable the OpenDLA backend, and build the example binaries as shown above.
5. Run the provided classification or detection examples, or convert your own Tengine models to the OpenDLA format.

Limitations

• Only the small NVDLA configuration is provided; larger configurations (medium, large) are not included.
• The demos target 8‑bit quantised models; full‑precision or mixed‑precision support would require additional work.
• Performance is bounded by the resources of the specific Zynq device you use; on low‑end Zynq‑7000 boards the accelerator may be resource‑constrained.

---

All information above is taken directly from the repository’s README.