OpenBMB/MiniCPM-Robot

A Smarter and Faster On-Device AI Brain for Robots

What it solves

MiniCPM-Robot provides on-device AI brains for robots to handle real-world perception, decision-making, and action. It addresses the need for efficient, low-latency models that can perform complex manipulation and target tracking directly on robot hardware without relying on heavy cloud computing.

How it works

The project consists of two primary model families:

  • MiniCPM-RobotManip: A 1.5B parameter Vision-Language-Action (VLA) model for generalist robot manipulation. It uses streaming inference to maintain a visual context memory (up to 1 minute) without the high computational cost of recomputing historical frames. It also employs visual token compression to reduce latency.
  • MiniCPM-RobotTrack: A 0.9B parameter policy for embodied target tracking. It utilizes a quality-driven self-evolving data pipeline and DAgger (Dataset Aggregation) to improve performance in difficult scenarios like target crossings and occlusions. It is optimized for end-to-end execution on the Unitree Go2 robot, integrating visual capture, encoding, and action generation.

Who it’s for

It is designed for robotics researchers and developers building embodied AI agents, specifically those targeting on-device deployment on hardware like the Unitree Go2 EDU or using simulators like LIBERO, CALVIN, and Habitat-Sim.

Highlights

  • Efficient Manipulation: A unified 1.5B generalist policy that outperforms larger models on several benchmarks while maintaining low latency (120ms per decision step on H100).
  • On-Device Tracking: The first fully on-device embodied target tracker (0.9B) capable of running at 5+ FPS on Unitree Go2 EDU.
  • Streaming Context: Supports up to 60 frames of history with significantly reduced TFLOPs compared to traditional recomputation.
  • End-to-End Optimization: Jointly optimizes the entire pipeline from visual capture to command execution for real-time robot control.

Related

  • Project
    OpenBMB/MiniCPM

    MiniCPM5-1B is a high-performance 1B-parameter dense LLM designed for on-device deployment, featuring hybrid reasoning capabilities and SOTA performance in coding and tool use.

  • Project
    OpenBMB/MiniCPM-V-Apps

    A collection of on-device mobile demos for the MiniCPM-V family of multimodal models, enabling local AI inference on iOS, Android, and HarmonyOS.

  • Project
    dexmal/opendm

    OpenDM is a Vision-Language-Action (VLA) model framework for open-world robot control, providing tools for training, fine-tuning, and deploying models that can handle complex, multi-embodiment tasks.

  • Project
    OpenBMB/MiniCPM-V

    A series of pocket-sized multimodal LLMs designed for efficient on-device deployment, enabling high-performance image, video, and real-time omnimodal interaction on mobile phones.

  • Project
    OpenSQZ/MiniCPM-V-CookBook

    A comprehensive implementation guide and deployment cookbook for the MiniCPM series, enabling the integration of vision, speech, and text AI capabilities across edge devices and enterprise servers.