MrGiovanni/ModelsGenesis

What it solves

Models Genesis addresses the scarcity of manually annotated 3D medical images (like CT and MRI scans), which are typically required to train high-performing deep learning models for segmentation and classification.

How it works

It uses a self-supervised learning framework to pre-train 3D models on unlabeled data. The model learns the underlying anatomy by performing image restoration proxy tasks, where it must reconstruct volumes that have been deliberately distorted. These distortions include:

• Non-linear intensity transformation
• Local pixel shuffling
• Out-painting
• In-painting

Once pre-trained, these "generic autodidactic models" serve as a foundation that can be fine-tuned for specific downstream medical imaging tasks.

Who it’s for

Medical imaging researchers and clinicians who need to train accurate 3D analysis models but have limited labeled datasets.

Highlights

• Self-supervised learning: Learns visual representations without any manual labels.
• 3D-native: Specifically designed for 3D volumes rather than relying on 2D ImageNet-based transfer learning.
• Proven performance: When combined with nnU-Net, it achieved top rankings in liver/tumor and hippocampus segmentation on the Medical Segmentation Decathlon.
• Flexible implementation: Provided in both PyTorch and Keras.