VIAME/VIAME

Video and Image Analytics for Multiple Environments

VIAME – A DIY Computer‑Vision Toolkit

What it is – VIAME (Vision for Intelligent Analytic Media Extraction) is an open‑source computer‑vision framework that bundles a large collection of algorithms, pipelines, and user interfaces for building AI‑powered video and image analysis applications. It started in the marine‑science community for species analytics but has grown into a generic, extensible toolkit.

Core ideas

  • Pipeline‑centric: C++/Python nodes are wired together in a multi‑threaded pipeline (built on Kitware’s KWIVER).
  • Plug‑in architecture: algorithms (detectors, trackers, filters, measurement tools, etc.) are provided as plugins and can be mixed with custom code.
  • Dual UI: a desktop GUI (VIVIA/DIVE) for annotation, training and visual inspection, plus a web‑based version for remote use.
  • End‑to‑end workflow: from raw video → data annotation → model training → inference → evaluation → search/mosaicing.

Key capabilities

Capability What you can do
Object detection & tracking Run pre‑trained detectors, train new models, and generate object tracks across video streams.
Data annotation Interactive tools (desktop and web) for drawing bounding boxes, polygons, and stereo measurements.
Multi‑camera & stereo Fuse feeds, measure real‑world size, and produce mosaics.
Image enhancement Apply filters, de‑noising, and contrast improvements before analysis.
Rapid model generation & search Build a model on a small labeled set and use it to query large video archives.
Evaluation & scoring Compare detections/tracks against ground‑truth CSV/COCO/JSON formats.
Extensible pipelines Combine any mix of C++ or Python processes; add new plugins via the plugins/ directory.

How to get it

  • Pre‑built binaries – Windows .zip or Linux .tar.gz packages (GPU‑enabled or CPU‑only) are linked in the README.
  • Docker – Pull kitware/viame:gpu-algorithms-default for a headless command‑line container; full web/desktop images are also provided.
  • Python package – pip install viame (Python 3.10‑3.14). The package installs a viame CLI that wraps the core tools and can download add‑on models.
  • Build from source – Clone the repo, initialise submodules, and run CMake with optional flags (e.g., VIAME_ENABLE_PYTORCH, VIAME_ENABLE_CUDA). See the Quick Build section for the exact commands.

Typical command‑line workflow

# set up environment (for binary or source installs)
source /opt/noaa/viame/setup_viame.sh

# run a pipeline on a video
viame run my_pipeline.pipe video.mp4

# train a detector on a labelled dataset
viame train data/ train_detector.conf

# evaluate results
viame score detections.csv groundtruth.csv

# list and install extra models
viame add-ons

The same pipelines can be invoked from Python via the viame package or by importing the underlying KWIVER classes.

Installation highlights

  • Requires ~8 GB disk space.
  • GPU acceleration needs an NVIDIA GPU with ≥8 GB VRAM and drivers ≥570.65.
  • Supported OS: Windows 7/8/10/11 (64‑bit), Linux (RHEL, CentOS, Ubuntu), and limited MacOS support for the web tools.
  • Optional dependencies (OpenCV, VXL, PyTorch, CUDA, TensorRT, ONNX, TensorFlow, Darknet, MATLAB) are toggled with CMake flags.

Documentation & community

  • Quick‑Start guide and full manual are hosted at https://viame.github.io/VIAME/.
  • Example pipelines, tutorials, and plugin templates live under the examples/ and plugins/ directories.
  • A public annotation archive and web annotator are available at https://viame.kitware.com/.
  • The project is BSD‑3 licensed; citations and acknowledgements are listed in docs/citations.md and docs/acknowledgements.md.

Who might use it

  • Researchers needing a ready‑made toolbox for video analytics (e.g., wildlife monitoring, surveillance, industrial inspection).
  • Developers building custom AI pipelines who want a C++/Python hybrid framework with GPU support.
  • Teams that require both a GUI for labeling and a headless CLI for batch processing.

All the information above is taken directly from the repository’s README; no additional features have been inferred.

Related

  • Project
    vllm-project/vime

    Vime is an open‑source framework that combines Megatron‑LM training with vLLM inference to enable high‑performance reinforcement‑learning fine‑tuning of large language models. It offers a unified data‑flow loop, flexible custom generation hooks, and support for many modern LLMs (Qwen, DeepSeek, LLaMA 3).

  • Project
    SamurAIGPT/Vibe-Workflow

    Vibe Workflow is an open‑source, MIT‑licensed node‑based visual editor for building generative image and video pipelines. It consists of a Next.js front‑end, a FastAPI back‑end that calls MuAPI’s generative models, and a reusable workflow‑builder UI library. You can self‑host it (Docker or manual) or use the hosted SaaS version, and you can extend it with custom nodes to call any AI model or external API.

  • Dispatch
    vime RL Framework Release

    vLLM introduces vime, an open-source RL post-training framework that integrates Megatron training with vLLM inference to provide a stable and efficient pipeline for LLM reinforcement learning.

  • Project
    open-gigaai/giga-models

    GigaModels is an open‑source Python toolbox that wraps dozens of pre‑trained vision, diffusion and multimodal models (e.g., Grounding DINO, Depth Anything, GigaBrain‑0, Pi0, Cosmos‑Predict2.5) behind a unified `load_pipeline` API. It supports both inference and training, offers ready‑to‑run scripts, and is installable via conda + pip.

  • Project
    OpenBMB/UltraRAG

    UltraRAG is an open‑source, low‑code framework for building, debugging, and deploying Retrieval‑Augmented Generation pipelines. It uses the Model Context Protocol to treat each component (retriever, LLM, etc.) as a plug‑and‑play server, orchestrated by YAML configuration. A built‑in web IDE provides visual pipeline editing and one‑click conversion to interactive chat demos. Install via the fast `uv` package manager or Docker, and use it for research experiments, product prototypes, or enterprise RAG services.