zuoqing1988/ZQCNN

What it solves

ZQCNN is a lightweight inference framework designed to run deep learning models efficiently across multiple platforms, including Windows, Linux, and ARM-Linux. It specifically addresses the need for high-performance model execution on CPUs without requiring heavy dependencies like MKL, while providing ready-to-use implementations for computer vision tasks.

How it works

The framework utilizes a custom GEMM (General Matrix Multiply) kernel called ZQ_GEMM, written in assembly and intrinsics to maximize throughput on x86 and ARM architectures. It supports various backends, allowing users to switch between ZQ_GEMM, MKL, or OpenBLAS. The framework can also compile models directly into code via a model2code tool to eliminate external file loading overhead.

Who it’s for

It is intended for developers who need to deploy AI models on edge devices or desktop environments where minimal dependencies and high CPU inference speed are critical, particularly for face detection and recognition applications.

Highlights

• Cross-Platform Support: Runs on Windows, Linux, and ARM-Linux.
• Custom Optimized Kernels: Includes ZQ_GEMM for high-performance matrix multiplication without needing MKL.
• Integrated CV Demos: Provides pre-trained models and demos for MTCNN face detection, SSD object detection, face recognition (MobileFaceNet), and OCR.
• Flexible Backend: Supports dynamic switching between different BLAS libraries for performance tuning.