apple-aiml-research/ml-sharp

What it solves

SHARP enables photorealistic view synthesis from a single image. It solves the problem of creating high-resolution, 3D-consistent views of a scene from one photograph quickly and with metric accuracy, avoiding the long processing times typically associated with 3D reconstruction.

How it works

The system uses a neural network to perform a single feedforward pass that regresses the parameters of a 3D Gaussian representation (3D Gaussian Splatting) of the scene. This process takes less than a second on a standard GPU. The resulting 3D representation is metric, meaning it has absolute scale, and can be rendered in real time to produce images from nearby viewpoints.

Who it’s for

This tool is designed for researchers and developers working in computer vision, 3D scene reconstruction, and photorealistic rendering who need fast, zero-shot generalization for monocular view synthesis.

Highlights

• Extreme Speed: Generates 3D representations in less than a second via a single feedforward pass.
• High Fidelity: Reduces LPIPS and DISTS errors significantly compared to previous state-of-the-art models.
• Metric Scale: Supports absolute scale for accurate metric camera movements.
• Broad Compatibility: Produces .ply files compatible with various public 3D Gaussian Splatting renderers.