zju3dv/snake

What is Deep Snake?

Deep Snake is an open‑source implementation of the Deep Snake model described in the CVPR 2020 paper “Deep Snake for Real‑Time Instance Segmentation.” It is a computer‑vision system that takes an image and outputs precise instance masks (the exact outlines of each object) in real time. The method improves on classic mask‑based approaches by iteratively refining object contours with a learnable “snake” (active‑contour) module, achieving state‑of‑the‑art results on benchmarks such as COCO, Cityscapes, KITTI and SBD.

Key Features (as documented in the README)

Feature	Details
Pre‑trained models	Links to Google‑Drive folders containing weights for Cityscapes, KITTI (referred to as Kins), and SBD datasets.
Evaluation scripts	`run.py` can evaluate on the COCO evaluator, the official Cityscapes evaluator, or the KITTI/SBD validation sets via simple command‑line flags.
Speed testing	The same `run.py` script can be invoked with `--type network` to report inference speed for a given configuration.
Visualization	Built‑in visualisation (`--type visualize`) draws the predicted masks on the original images; example outputs are shown in the README.
Demo mode	One‑line demo for a single image or a folder of images (`--type demo`).
Training pipelines	Two‑stage training for Cityscapes (first train a detector, then jointly train detector + snake). Single‑stage training scripts are provided for KITTI and SBD.
TensorBoard support	Logs are written to `data/record/*` and can be inspected with TensorBoard to monitor loss curves and other metrics.
Config‑driven	All experiments are driven by YAML config files (e.g., `configs/city_rcnn_snake.yaml`).

How to Get Started

Clone the repo

git clone https://github.com/zju3dv/snake.git
cd snake

Install dependencies – see INSTALL.md for the exact package list (PyTorch, CUDA, OpenCV, etc.).
Download a pre‑trained model for the dataset you want to try (Cityscapes, KITTI, or SBD) and place it under the path indicated in the README, e.g.
```
$ROOT/data/model/rcnn_snake/long_rcnn/197.pth   # Cityscapes
```
Run a quick demo on an image folder:
```
python run.py --type demo --cfg_file configs/sbd_snake.yaml \
             demo_path demo_images ct_score 0.3
```
The script will output the original image with the predicted instance contours overlaid.

Typical Workflows

Goal	Command (example)
Evaluate on Cityscapes (COCO evaluator)	`python run.py --type evaluate --cfg_file configs/city_rcnn_snake.yaml`
Evaluate on Cityscapes (official evaluator)	`python run.py --type evaluate --cfg_file configs/city_rcnn_snake.yaml test.dataset CityscapesVal`
Measure inference speed	`python run.py --type network --cfg_file configs/city_rcnn_snake.yaml`
Visualise results	`python run.py --type visualize --cfg_file configs/city_rcnn_snake.yaml test.dataset CityscapesVal ct_score 0.3`
Train detector only (Cityscapes)	`python train_net.py --cfg_file configs/city_ct_rcnn.yaml model rcnn_det`
Train detector + snake (Cityscapes)	`python train_net.py --cfg_file configs/city_rcnn_snake.yaml model rcnn_snake det_model rcnn_det`
TensorBoard	`tensorboard --logdir data/record/rcnn_snake` (or `.../snake` for the snake‑only logs)

Who Might Use This?

Researchers exploring contour‑based instance segmentation or looking for a strong baseline to compare against.
Practitioners needing fast, accurate mask predictions for autonomous‑driving datasets (Cityscapes, KITTI) or general object segmentation (COCO, SBD).
Students who want a concrete example of how to integrate a custom module (the snake) into a detection pipeline.

Citation

If you use the code in a publication, cite the original paper:

@inproceedings{peng2020deep,
  title={Deep Snake for Real-Time Instance Segmentation},
  author={Peng, Sida and Jiang, Wen and Pi, Huaijin and Li, Xiuli and Bao, Hujun and Zhou, Xiaowei},
  booktitle={CVPR},
  year={2020}
}

Bottom line: The repository provides a ready‑to‑run implementation of a real‑time instance‑segmentation algorithm that leverages a learnable active‑contour (snake) model. With pre‑trained weights, clear evaluation/visualisation scripts, and training pipelines, it is a practical resource for anyone working on modern segmentation tasks.