NVIDIA/torch-harmonics

What it solves

torch-harmonics provides a framework for differentiable signal processing on the sphere, enabling the integration of spherical data into machine learning models. It solves the difficulty of performing Fourier-like transforms and convolutions on spherical surfaces without losing differentiability or efficiency, which is critical for solving partial differential equations (PDEs) and training neural operators on spherical domains.

How it works

The library implements the Spherical Harmonic Transform (SHT), which generalizes the Fourier transform to the sphere. It uses quadrature rules to project signals onto associated Legendre polynomials and Fast Fourier Transforms (FFTs) for the harmonic basis. To ensure compatibility with deep learning, it is built using PyTorch primitives, making all operations fully differentiable. It also introduces local discrete-continuous (DISCO) convolutions and spherical attention mechanisms that use geodesic distance to determine neighborhoods, ensuring the operations are approximately equivariant.

Who it’s for

Researchers and engineers working with spherical data, such as those developing differentiable PDE solvers, Spherical Fourier Neural Operators (SFNO), or models for depth estimation and semantic segmentation on the sphere.

Highlights

• Differentiable SHT: Fully differentiable implementations of spherical, vector spherical, and discrete-continuous convolutions.
• GPU Acceleration: Optimized CUDA extensions for high-performance computation on NVIDIA GPUs.
• ** uma Distributed Computing**: Supports spatially distributed quadrature to enable scalable model-parallelism.
• Spherical Attention: Generalizes the attention mechanism to the sphere using geodesic distances for neighborhood determination.
• Broad Application: Used for tasks like solving shallow water equations, depth estimation, and semantic segmentation on spherical grids.