zsylvester/segmenteverygrain

What it solves

It automates the detection and segmentation of grains (or grain-like objects) in images, which is essential for analyzing grain size and shape in geomorphology and sedimentary geology. It addresses the limitations of using the Segment Anything Model (SAM) alone, such as slow processing in "everything" mode and the production of overlapping masks or background noise.

How it works

The tool uses a two-step process: first, a patch-based convolutional neural network (U-Net style) creates an initial coarse segmentation. This first pass is then used to generate precise prompts for SAM 2.1, which produces high-quality final outlines of the grains. The system also includes an interactive editing interface (GrainPlot) for manual quality control, allowing users to delete, merge, or add grains, and a grain_utils module for extracting and clustering grains for classification.

Who it’s for

Geologists, geomorphologists, and researchers who need to accurately measure grain characteristics from photographic images.

Highlights

• Hybrid ML Approach: Combines a U-Net for fast prompting and SAM 2.1 for high-precision segmentation.
• Interactive QC: A dedicated interface for adding, deleting, and merging grains via mouse clicks and keyboard shortcuts.
• Fine-tuning: Ability to actually fine-tune the base U-Net model on new image types to improve detection accuracy.
• Analysis Tools: Built-in utilities for cropping individual grains, extracting features using pre-trained CNNs (VGG16, ResNet50, InceptionV3), and clustering them using K-means or DBSCAN.