kornia/kornia

What it solves

Kornia provides a differentiable computer vision library that allows traditional image processing and geometric vision algorithms to be integrated directly into deep learning pipelines. It solves the problem of bridging the gap between classical computer vision and modern AI by making these operations differentiable, enabling them to be used with auto-differentiation and GPU acceleration within the PyTorch ecosystem.

How it works

Built on top of PyTorch, Kornia implements a wide array of vision operators as tensors. This allows for batch transformations and the use of gradients for optimization. The library is organized into several key components:

• Differentiable Image Processing: Includes filters (Gaussian, Sobel), transformations (Affine, Homography), and enhancements (CLAHE, Gamma Correction).
• Advanced Augmentations: Provides pipelines for complex data augmentation, including automatic augmentation strategies like RandAugment.
• AI Models: Integrates pre-trained models for tasks such as face detection (YuNet), feature matching (LoFTR, LightGlue), and segmentation (SAM).
• Geometry Core: Handles camera models, calibration, stereo vision, and 3D transformations.

Who it’s for

Kornia is designed for AI researchers and developers working on computer vision, specifically those who need to integrate classical vision algorithms into neural network training or inference workflows.

Highlights

• Differentiable Operators: Over 500 operations that support auto-differentiation.
• GPU Acceleration: Seamless integration with PyTorch for high-performance processing.
• Comprehensive Geometry: Robust support for epipolar geometry, homographies, and Lie groups.
• Half-Precision Support: Extensive testing and support for float16 and bfloat16 dtypes.
• ONNX Integration: Ability to chain and export ONNX models using ONNXSequential.