Gemini Robotics 1.5 and Gemini Robotics-ER 1.5 Release

Google DeepMind has released Gemini Robotics 1.5 and Gemini Robotics-ER 1.5, two specialized models designed to transition AI agents from digital environments into the physical world. By splitting the cognitive load between high-level planning and low-level motor execution, these models enable robots to perceive, plan, and act to solve complex, multi-step tasks with greater transparency and generalization.

The Agentic Framework: Orchestration and Execution

Google DeepMind utilizes a two-model agentic framework to handle the complexity of real-world tasks, such as sorting objects based on local recycling guidelines.

Gemini Robotics-ER 1.5 (Embodied Reasoning)

Gemini Robotics-ER 1.5 serves as the "high-level brain" of the system. It is a vision-language model (VLM) optimized for embodied reasoning that orchestrates a robot's overall activities. Its primary capabilities include:

  • Planning and Decision Making: Creating detailed, multi-step plans to complete a mission.
  • Tool Integration: Natively calling digital tools, such as Google Search, or third-party user-defined functions to gather necessary information.
  • Spatial Understanding: Achieving state-of-the-art performance across spatial understanding benchmarks, including pointing, image question answering, and video question answering.
  • Monitoring: Estimating its own success and progress throughout a task.

Gemini Robotics 1.5 (Vision-Language-Action)

Gemini Robotics 1.5 is a vision-language-action (VLA) model that converts the natural language instructions provided by the ER model into direct motor commands. Unlike traditional VLA models that simply translate plans into movement, Gemini Robotics 1.5 "thinks before acting," generating an internal sequence of reasoning and analysis in natural language. This allows the robot to:

  • Deconstruct Complex Tasks: Break longer tasks into simpler, shorter segments for more reliable execution.
  • Increase Transparency: Explain its thinking process in natural language, making its decision-making process visible to users.
  • Improve Robustness: Better generalize to new tasks and adapt to changes in the environment.

Cross-Embodiment Learning and Generalization

Gemini Robotics 1.5 addresses the challenge of transferring skills between robots of different shapes, sizes, and degrees of freedom. The model demonstrates the ability to learn across embodiments, meaning motions learned on one robot can be transferred to another without requiring model specialization for each new hardware configuration.

Google DeepMind observed this capability across several platforms: tasks trained on the ALOHA 2 robot were successfully executed by the Apptronik Apollo humanoid robot and the bi-arm Franka robot, and vice versa.

Performance Benchmarks and Safety

Spatial Reasoning Benchmarks

Gemini Robotics-ER 1.5 was evaluated on 15 academic benchmarks, achieving state-of-the-art aggregated performance. These benchmarks include:

  • Embodied Reasoning Question Answering (ERQA)
  • Point-Bench
  • RefSpatial, RoboSpatial-Pointing, Where2Place, BLINK, CV-Bench, EmbSpatial, MindCube, RoboSpatial-VQA, SAT, Cosmos-Reason1, Min Video Pairs, OpenEQA, and VSI-Bench.

Semantic Safety and the ASIMOV Benchmark

To ensure responsible deployment in human-centric environments, Google DeepMind updated the ASIMOV benchmark, a collection of datasets for evaluating semantic safety. Gemini Robotics-ER 1.5 shows state-of-the-art performance on this benchmark.

Safety is implemented through a holistic approach:

  • High-level reasoning: The model thinks about safety constraints before acting.
  • Policy alignment: Dialogue is aligned with Gemini Safety Policies.
  • Low-level triggers: The system can trigger on-board safety sub-systems, such as collision avoidance, when necessary.

Availability

  • Gemini Robotics-ER 1.5: Available to developers via the Gemini API in Google AI Studio.
  • Gemini Robotics 1.5: Currently available to select partners.

Sources