Gemini Robotics ER 2 release notes / what's new

Google DeepMind has introduced Gemini Robotics ER 2, a high-level "embodied reasoning" model designed to serve as a brain for robots. The model enables robots to interact with humans, understand physical environments, and plan multi-step tasks, which it then hands off to lower-level vision-language-action (VLA) models for motor execution.

Real-Time Task Orchestration and Agentic Capabilities

Gemini Robotics ER 2 acts as a physical agent that orchestrates complex, multi-step workflows, allowing robots to self-correct and generalize to novel situations. Developers can integrate low-level control interfaces—such as navigation APIs or VLA models—as tools that the model can call natively, alongside tools like Google Search.

Key improvements in orchestration include:

  • Latency Reduction: Integration with the Gemini Live API via a bidirectional streaming endpoint minimizes "stop-and-think" pauses, enabling fluid execution of tasks.
  • Performance Gains: The model consistently outperforms Gemini Robotics ER 1.6 in tool orchestration across real VLA, simulated VLA, and human tele-operation modes.
  • Practical Application: In a demonstration with Boston Dynamics' Spot, the model orchestrated Spot APIs for navigation and manipulator movement to fetch objects based on natural language commands.

Temporal Intelligence and Video Understanding

Gemini Robotics ER 2 introduces significant advancements in "temporal intelligence," allowing robots to track progress and identify precise moments for task transitions.

Continuous Progress Classification

The model can track progress toward task completion by assigning frames in a video feed into five progress levels (0-20% to 80-100%). This allows robots to adjust actions in real-time or retry failed steps without restarting the entire workflow. Gemini Robotics ER 2 achieves 57.4% accuracy in progress classification, surpassing previous generation and competing frontier models.

Precision Moment-Finding

To determine exactly when to switch tasks (e.g., when to stop pouring a liquid), the model utilizes moment-finding. It achieves 91.3% accuracy and a 0.96s mean absolute distance. This precision is delivered at 4x the execution speed of larger model categories, providing the sub-second latency required for safe real-world operation.

Multi-Robot Collaboration and Spatial Intelligence

Gemini Robotics ER 2 enables diverse robots to communicate via a shared semantic understanding to complete tasks that a single robot cannot perform alone. This collaboration has been demonstrated using Apptronik's Apollo 2 and the Franka F3 Duo.

Additionally, the model improves general spatial intelligence across three core areas:

  • Success/Failure Detection: Now utilizes raw video feeds instead of static snapshots to detect mid-execution failures such as slips or spills.
  • General Instrument Reading: The model can now read 10 different types of instruments, including digital displays, linear scales, and liquid thermometers.
  • Enhanced Spatial VQA: Improved Visual Question Answering capabilities through advanced multi-modal understanding.

Safety and Human Proximity

Gemini Robotics ER 2 is described as Google DeepMind's safest embodied reasoning model to date. It shows significant gains in Safety Instruction Following and Human Proximity benchmarks, specifically demonstrating the ability to halt a humanoid robot when a person is nearby and resume only once the area is clear.

Google DeepMind has also introduced a new benchmark to evaluate a foundation model's capacity to act as a safe VLA orchestrator by monitoring the environment, enforcing safety constraints, and assessing physical feasibility.

Sources