Mosi-AI/RoboICL

Embodied In-Context Learning for GPT-6 Astra

What it solves

RoboICL addresses the challenge of enabling robots to learn new tasks without requiring expensive retraining or specialized robot-control models. It allows a general-purpose vision-language model to control a robot by adapting to tasks on the fly using only a few demonstrations or its own execution history.

How it works

The framework connects a vision-language model (specifically GPT-6 Astra) to a robot simulator. It uses two main sources of context: a demonstration context (recorded robot trajectories) and an interaction memory (the model's own past actions and feedback). These are presented to the model using a shared grammar that maps observations to actions. The system maintains a bounded memory, keeping key historical interactions while prioritizing the most recent results for immediate correction.

Who it’s for

This project is for researchers and developers working in robotics and embodied AI who want to experiment with using large-scale multimodal models for robot control. It is designed for those using the RoboDojo benchmark and Isaac Sim environments.

Highlights

  • Zero-shot and one-shot learning capabilities without parameter updates.
  • Uses a shared interaction grammar for consistent observation-action loops.
  • Bounded memory management to balance long-term context with immediate feedback.
  • Includes a complete runner for the RoboDojo benchmark with support for dual-arm Cartesian actions.
  • Orchestrates separate Python environments for simulation and policy execution to handle complex dependency requirements.

Related

  • Project
    syswonder/robonix

    Robonix is an open‑source, agentic operating system for embodied AI. It translates natural‑language commands into a Robot Task Description Language (RTDL) program using LLM/VLM models, then executes the program on a wide range of real or simulated robots. The system is built on layered abstractions—primitive, service, skill, task—registered via versioned contracts, enabling hardware‑software decoupling and reuse across heterogeneous platforms. The README provides a quick‑start that installs Rust, uv, Docker, builds the `rbnx` CLI, runs a Webots simulator, and launches the system with an OpenAI‑compatible VLM endpoint. A rich package catalog lists dozens of supported robot bodies, both physical and simulated. The project is licensed under Mulan PSL‑2.0.

  • Project
    OpenBMB/DeepThinkVLA

    DeepThinkVLA is a Vision‑Language‑Action model that inserts an explicit chain‑of‑thought reasoning step before generating robot actions. Using a hybrid decoder (autoregressive reasoning → bidirectional parallel action generation), the authors train on a newly curated embodied CoT dataset and then refine with outcome‑driven reinforcement learning. The model achieves 97 % average success on the LIBERO benchmark, runs at only 0.175× the latency of a pure autoregressive baseline, and transfers zero‑shot to the newer LIBERO Plus suite.

  • Project
    open-gigaai/giga-brain-0

    GigaBrain‑0.7 is an open‑source, large‑scale vision‑language‑action model for robots. It is pretrained on 37 k h of heterogeneous embodied data, supports eight robot embodiments, and ships with training scripts, benchmark suites (VLM‑Benchmark, RoboColiseum, RoboTwin 2.0, EBench), and ready‑to‑run inference servers that integrate with ROS.

  • Project
    2toinf/X-VLA

    X‑VLA is a transformer model that unifies vision, language, and robot proprioception to predict actions across many robot types. It uses learnable soft prompts to specialize a shared backbone for each embodiment, runs as a REST inference server, and provides many pre‑trained checkpoints (foundation and robot‑specific) on Hugging Face. The repo includes installation scripts, server/client code, and instructions for fine‑tuning on custom robot datasets.

  • Project
    Robbyant/lingbot-vla-v2

    LingBot‑VLA 2.0 is an open‑source Vision‑Language‑Action foundation model for robots. It unifies many robot embodiments into a 55‑dim action vector, uses MoE layers and dual‑query distillation (depth + future video) to learn predictive dynamics, and is trained on ~60 k h of robot and egocentric video data. The repo provides conda‑based installation, scripts to download 6 B‑parameter checkpoints, tools for data preparation, post‑training on custom datasets, and evaluation on the RoboTwin benchmark and real robots. Reported results show noticeable gains over the 1.0 version on bimanual and mobile manipulation tasks.