10x-Engineers/Infinite-ISP

What it solves

Infinite-ISP provides a full-stack development platform for creating and tuning Image Signal Processors (ISPs). It addresses the fragmentation of open-source ISP development by centralizing various camera pipeline modules into a single platform, allowing developers to implement, compare, and tune both conventional and deep learning-based image processing algorithms.

How it works

The platform operates as a multi-stage pipeline that converts RAW sensor data into RGB images. It consists of several layers of implementation:

• Algorithm Design: A Python-based model used for high-level algorithm development.
• Reference Model: A fixed-point Python model that serves as the blueprint for hardware implementation.
• RTL Design: A Verilog implementation of the ISP for hardware synthesis.
• FPGA Integration: A framework and firmware specifically for the Xilinx Kria KV260 development board.
• Tuning Tool: A standalone Python application for calibrating ISP parameters for different sensors.
• Linux Stack: Drivers and an application stack to integrate the ISP into Linux platforms.

Who it’s for

It is designed for ISP developers, hardware engineers, and researchers working on camera pipelines, sensor calibration, and FPGA-based image processing.

Highlights

• Comprehensive Pipeline: Includes modules for dead pixel correction, black level correction, Bayer noise reduction, white balance, demosaicing, and color space conversion.
• 3A Algorithms: Implements Auto Exposure (AE) and Auto White Balance (AWB) using methods like Grey World and PCA.
• Hardware-Ready: Provides a path from Python prototyping to RTL and FPGA deployment.
• Flexible Configuration: Uses YAML-native pipeline editing and configuration files to tune sensor-specific parameters.
• Performance Metrics: Includes tools for evaluating image quality using PSNR and SSIM metrics.