MIC-DKFZ/nnInteractive

nnInteractive is a framework for 3D interactive segmentation, supporting intuitive prompts like points, scribbles, bounding boxes, and lasso. Trained on 120+ diverse 3D datasets, it sets a new standard in accuracy, usability, and adaptability for clinical and research applications.

What it solves

nnInteractive is a framework for 3D promptable segmentation, specifically designed for 3D medical images (CT, MRI, PET, 3D Microscopy). It addresses the limitations of 2D foundation models like SAM, which often struggle with volumetric awareness and domain shifts when applied to 3D medical data. The project provides an open-set segmentation method that allows users to refine 3D segmentations through intuitive 2D interactions.

How it works

The system leverages a model trained on over 120 diverse volumetric 3D datasets. It allows users to provide various prompts—including points, scribbles, 2D bounding boxes, and a novel lasso prompt—to generate and incrementally refine 3D segmentations.

Technically, it is split into two components:

  • Local Inference Engine: A full stack requiring PyTorch and an NVIDIA GPU for local processing.
  • Remote Client/Server: A lightweight client (nninteractive-client) that can drive a remote GPU-powered server over HTTP, allowing machines without GPUs to perform complex 3D segmentation.

Who it’s for

Researchers, developers, and clinical practitioners working with 3D medical imaging who need accurate, interactive segmentation tools. It is designed for integration into Python-based workflows or existing imaging platforms like Napari, MITK, 3D Slicer, ITK-SNAP, and OHIF.

Highlights

  • Diverse Prompting: Supports points, scribbles, 2D bounding boxes, and lasso prompts.
  • Volumetric Awareness: Specifically trained for 3D medical modalities (CT, MRI, PET, etc.).
  • Client-Server Architecture: Enables remote GPU inference via a lightweight client.
  • Broad Integration: Official and community-driven plugins for major medical image viewers.
  • State-of-the-Art: Won 1st place in the CVPR 2025 Challenge on Interactive 3D Segmentation.

Related

  • Project
    facebookresearch/pixio

    Pixio is a vision encoder optimized for dense prediction tasks like depth estimation and semantic segmentation, achieved through enhanced pixel reconstruction pre-training.

  • Project
    open-edge-platform/geti_v2

    An end-to-end computer vision platform that uses active learning and smart annotations to build and deploy optimized AI models with minimal data.

  • Project
    Seeed-Studio/OSHW-reCamera-Series

    A hardware documentation and design-resource hub for the reCamera series of AI-powered vision modules with integrated NPUs for edge AI.

  • Project
    cre185/InstantSfM

    A GPU-native Structure from Motion (SfM) pipeline that accelerates the process of reconstructing 3D structures from 2D images, including integrated support for 3D Gaussian Splatting.