jatinchowdhury18/RTNeural

What it solves

RTNeural is a lightweight C++ inferencing engine designed for real-time systems, with a specific focus on real-time audio processing. It allows developers to run pre-trained neural networks in environments where low latency and high performance are critical.

How it works

The library takes neural networks trained in frameworks like TensorFlow or PyTorch, imports their weights via JSON files, and executes the forward pass (inference) in C++. It offers two primary APIs:

• Dynamic API: Creates the inferencing engine at run-time from a JSON file.
• Compile-Time API: Defines the model architecture at compile-time, which significantly improves performance by fixing the network structure.

To optimize performance, it supports three backends: Eigen, xsimd, and the C++ STL, and can be built with AVX SIMD extensions.

Who it’s for

It is intended for C++ developers building real-time applications, particularly those creating audio plugins (VST, AU, LV2) or embedded systems where neural network inference must happen in real-time.

Highlights

• Real-time optimized: Specifically tailored for low-latency audio processing.
• Flexible weight loading: Supports importing weights from TensorFlow and PyTorch.
• Broad layer support: Includes Dense, GRU, LSTM, Conv1D, Conv2D, and BatchNorm layers.
• Multiple backends: Choice of Eigen, xsimd, or STL for platform-specific performance tuning.
• Compile-time optimization: Ability to define models statically for maximum speed.