ashvardanian/NumKong

What it solves

NumKong is a portable, mixed-precision math and linear algebra library designed to eliminate "quiet overflow" and numerical instability common in standard BLAS libraries. It provides high-performance SIMD kernels that promote results to wider accumulators (e.g., Int8 $\to$ Int32) to ensure results stay in range, while maintaining a significantly smaller binary footprint than alternatives like PyTorch or NumPy.

How it works

The library implements over 2,000 SIMD kernels across 30+ backends for x86, Arm, RISC-V, LoongArch, Power, and WebAssembly. It leverages hardware-specific features like AMX and SME and supports 15+ numeric types, ranging from 4-bit integers to 128-bit complex numbers.

Key design choices include:

• Avoiding loop unrolling: Relying on modern CPU hardware to handle out-of-order execution and register renaming to keep the binary size small and avoid instruction-cache pressure.
• Thread-agnostic design: It does not manage its own thread pools, instead exposing row-range parameters so developers can integrate it into any parallelism model (e.g., Python's concurrent.futures) without thread oversubscription.
• Memory independence: It avoids hidden internal allocations, making it compatible with arbitrary allocators and alignment requirements.

Who it’s for

It is built for developers needing high-performance, numerically stable linear algebra across multiple languages (C++, Python, Rust, JS, Go, Swift) and platforms, particularly those working in robotics, edge AI, or real-time systems where deterministic latency and small binary sizes are critical.

Highlights

• Mixed-Precision Support: Handles 17 numeric types, including rare formats like 6-bit floats.
• Lighter Footprint: Binary size is 5-100x smaller than other BLAS-like alternatives.
• Broad Hardware Support: Optimized for x86, Arm, RISC-V, LoongArch, Power, and WebAssembly.
• Numerical Validation: Every kernel is validated against 118-bit extended-precision baselines.
• Multi-language Bindings: Native support for six major programming languages.