vccimaging/DeepLens

What it solves

DeepLens addresses the limitations of traditional commercial optical design software, which typically relies on manual GUI-based adjustments and finite difference gradients. It provides a differentiable framework that allows optical systems to be treated as learnable parameters within a PyTorch training loop, enabling the joint optimization of optics and neural networks.

How it works

DeepLens is a differentiable optical lens simulator that integrates various optical models—including ray tracing, wave propagation, and surrogate PSF networks—directly into a PyTorch computation graph. This allows gradients to flow through the optical simulation via autograd, enabling GPU-accelerated, batched simulations and end-to-end co-design of the optics, image signal processor (ISP), and reconstruction networks.

Who it’s for

It is designed for researchers and engineers in computational imaging and optical design who need to prototype custom optical systems, automate lens design, or generate synthetic datasets for image simulation.

Highlights

• Differentiable Engine: Uses analytic gradients via autograd for faster and more accurate optimization compared to finite differences.
• AI-Native: Designed to be scriptable by AI agents to perform optical reasoning and simulations (e.g., computing MTF).
• Multi-Model Support: Supports ray tracing, wave propagation, ray-wave hybrid models, and non-sequential polarization tracing.
• End-to-End Co-design: Integrates with frameworks like End2endImaging to optimize the entire camera pipeline from lens to neural reconstruction.