mapillary/seamseg

Seamless Scene Segmentation

What it solves

It addresses the challenge of "Panoptic Segmentation," which requires a model to provide a complete, pixel-level labeling for an image that identifies both the class (what the object is) and the specific instance (which individual object it is).

How it works

The project uses a CNN-based architecture implemented in PyTorch. It features a novel segmentation head that combines multi-scale features from a Feature Pyramid Network (FPN) with contextual information from a lightweight module similar to DeepLab. This allows the network to be trained end-to-end to predict precise labels for every pixel in a scene.

Who it’s for

This tool is designed for computer vision researchers and developers working on scene understanding, specifically those focusing on panoptic segmentation for complex environments like cityscapes.

Highlights

  • End-to-end training: The architecture is designed to be trained as a single unit for panoptic segmentation.
  • Multi-scale integration: Combines FPN features with contextual information for better accuracy.
  • Dataset support: Includes scripts to convert data from popular datasets like Cityscapes and Mapillary Vistas.
  • Comprehensive metrics: Supports evaluation using mAP, Panoptic Quality (PQ), and mIOU.

Related

  • Project
    qubvel-org/segmentation_models.pytorch

    A PyTorch library providing a high-level API for image semantic segmentation, featuring 12 architectures and over 800 pretrained encoders.

  • Project
    sstary/SSRS

    A PyTorch collection of deep learning models for remote sensing semantic segmentation, featuring single-modal, multimodal fusion, and unsupervised domain adaptation techniques.

  • Project
    jacobgil/pytorch-grad-cam

    A PyTorch library for explainable AI in computer vision that provides a wide range of pixel attribution methods to visualize and diagnose model predictions.

  • Project
    SegmentationBLWX/sssegmentation

    A PyTorch-based supervised semantic segmentation toolbox providing a unified framework and extensive model zoo for training and testing segmentation algorithms.

  • Project
    TissueImageAnalytics/tiatoolbox

    A computational pathology toolbox based on PyTorch that provides an end-to-end API for analyzing tissue images, from data loading to visualization.