PozzettiAndrea/ComfyUI-DepthAnythingV3

What it solves

This project integrates the Depth Anything V3 model into ComfyUI, allowing users to generate spatially consistent depth maps from images and videos. It solves the problem of recovering visual space and geometry from visual inputs, providing tools for both 2D depth mapping (for use in ControlNet) and 3D reconstruction.

How it works

It provides a set of custom nodes for ComfyUI that interface with various Depth Anything V3 model variants (ranging from Small to Giant). Depending on the chosen model, the system can perform monocular depth estimation, metric depth estimation, and multi-view attention to ensure depth consistency across video frames. It also includes specialized normalization modes (like V2-Style) and tools to convert depth maps into 3D point clouds or meshes.

Who it’s for

• AI Artists and Designers: Users of ComfyUI who need high-quality depth maps for ControlNet workflows.
• 3D Developers: Those looking to reconstruct 3D point clouds or meshes from single or multiple images.
• Video Creators: Users needing smooth, consistent depth estimation across video frames.

Highlights

• Multiple Model Variants: Supports a wide range of models from the lightweight DA3-Small (80M) to the high-fidelity DA3-Nested-Giant-Large (1.4B).
• 3D Reconstruction: Ability to generate 3D point clouds and meshes from visual inputs.
• Multi-View Consistency: Uses cross-attention features to maintain consistent depth across video frames.
• Specialized Outputs: Provides sky segmentation masks to filter out background noise in 3D reconstructions.
• Flexible Normalization: Offers different normalization modes to suit either 2D image generation or 3D spatial recovery.