v-modal/vmodal_sdk_robotics

Robotics vision SDK

What it solves

This SDK provides a crash-resilient data uplink for robotics and physical AI, specifically designed to move LeRobot dataset artifacts (videos, telemetry, and metadata) from a robot to a remote server without risking data loss during network instability or power failures.

How it works

The system uses a "handoff protocol" where a recorder closes files and publishes a .ready.json manifest. The SDK then:

  1. Verifies and Spools: It validates the manifest and copies the referenced files into a durable local spool (using SQLite in WAL mode for state tracking and atomic file operations for data).
  2. Independent Lanes: It processes uploads via separate async lanes for video and auxiliary data (telemetry/metadata) to ensure large video files don't block critical telemetry.
  3. Reliable Transport: It manages retries with exponential backoff and performs "restart reconciliation" to recover the state of uploads that were interrupted by a crash or power loss.
  4. Transport Abstraction: A narrow Transport interface allows the SDK to work with various backends (like S3, R2, or the V-Modal cloud) without requiring the recorder to depend on those clients.

Who it’s for

Robotics engineers and AI researchers using the LeRobot framework who need a reliable way to stream dataset captures to the cloud or on-prem storage from hardware that experiences intermittent connectivity or power cycles.

Highlights

  • Crash Resilience: Uses fsync, atomic renames, and SQLite WAL mode to ensure data survives process crashes and power loss.
  • Checksum-Keyed Spool: Every artifact is tracked by SHA-256 hashes to prevent corruption and ensure identity stability.
  • Lane Independence: Prevents telemetry starvation by separating video and auxiliary data flows.
  • Strict Handoff: Enforces a clear boundary where the recorder owns mutable files and the uploader owns immutable bytes.
  • Low Dependency Footprint: The core runtime requires only CPython and Fire, avoiding heavy dependencies like PyTorch or ROS on the robot.

Related

  • Project
    nvidia-cosmos/cosmos-rl

    A scalable Reinforcement Learning framework specialized for Physical AI applications, providing a decoupled architecture for large-scale training workloads.

  • Project
    GalaxyGeneralRobotics/Humanoid-GPT

    A GPT-style humanoid motion Transformer pre-trained on 2 billion frames to enable zero-shot whole-body motion tracking for humanoid robots like the Unitree G1.

  • Project
    luohongk/SuperVINS

    A real-time Visual-Inertial SLAM framework that replaces traditional handcrafted features with SuperPoint and LightGlue deep learning models to handle challenging imaging conditions.

  • Project
    sjtuyinjie/M3DGR

    M3DGR is a multi-sensor SLAM benchmark dataset and resilient framework designed to test and improve the robustness of ground robot localization and mapping in degraded environments.