vccimaging/DeepLens

What it solves

DeepLens addresses the limitations of traditional commercial optical design software, which typically rely on GUIs and finite difference methods. It provides a differentiable framework that allows optical systems to be treated as learnable parameters within a PyTorch training loop, enabling the joint optimization of optics and neural networks.

How it works

DeepLens is a differentiable optical lens simulator that integrates multiple optical models—including ray tracing, wave propagation, and surrogate PSF networks—directly into a PyTorch computation graph. This allows gradients to flow through the optical simulation via autograd, enabling GPU-accelerated, batched simulations and end-to-end co-design of the entire imaging pipeline (optics, sensor/ISP, and reconstruction networks).

Who it’s for

It is designed for researchers and engineers who need to prototype custom optical systems, perform automated lens design, or generate synthetic datasets for computational imaging.

Highlights

• Differentiable Engine: Uses analytic gradients via autograd for faster and more efficient optimization than finite differences.
• AI-Native: Designed to be scriptable by AI agents to perform optical reasoning and simulations.
• Multi-Model Support: Supports ray tracing, wave propagation, and hybrid ray-wave models for refractive and diffractive lens design.
• End-to-End Co-design: Enables joint optimization of the physical lens and the downstream image processing algorithms.