nv-tlabs/NKSR

What it solves

NKSR addresses the challenge of reconstructing high-quality 3D implicit surfaces from point clouds that are large-scale, sparse, and noisy. It specifically overcomes limitations in previous neural kernel representations by enabling the processing of massive scenes (spanning kilometers) and millions of points while remaining robust to noise.

How it works

The system builds upon Neural Kernel Fields (NKF) and employs several key technical improvements:

• Compactly Supported Kernels: Uses kernel functions that only act locally, allowing the use of memory-efficient sparse linear solvers to scale to large environments.
• Gradient Fitting: Implements a gradient fitting solve to ensure the reconstruction is robust against noise in the input data.
• Flexible Training: Learns from dense oriented points, allowing the model to mix training data from objects and scenes of vastly different scales.
• Out-of-Core Processing: Capable of handling very large scenes that exceed available memory by processing them in an out-of-core fashion.

Who it’s for

This tool is designed for researchers and developers working in 3D computer vision, robotics, and digital twinning who need to convert raw, noisy point cloud data (such as iPhone scans or LiDAR data) into clean 3D meshes.

Highlights

• High Scalability: Can reconstruct scenes spanning kilometers with millions of points on a single consumer GPU (RTX 3090).
• Rapid Performance: Capable of reconstructing millions of points in just a few seconds.
• Versatile Application: Proven effective across single objects, indoor scenes, and outdoor environments.
• Generalization: Maintains strong generalization capabilities, allowing it to work on new data with minimal training requirements.