EMERGE-Policy/EMERGE-Policy

What it solves

Traditional embodied intelligence uses a single end-to-end model for robot control. But long-horizon tasks need constant low-level perception, high-level planning, and fine-grained action generation. One model struggles to balance these competing demands, leading to confused reasoning, error accumulation, and task failure. EMERGE-Policy replaces the single-policy approach with a system where multiple specialized models collaborate, so intelligence emerges from their coordination rather than living inside one network.

How it works

EMERGE-Policy is a graph-structured agent runtime. A primary agent coordinates planning while several sub-agents handle perception, verification, and action. The system invokes external models as tools: OpenPI or Cosmos Policy generate actions, VGGT estimates geometry from images, and SAM3 performs image segmentation. These models run as persistent services in separate environments, and the agent orchestrates them asynchronously. You can interact through a web interface (recommended), a terminal UI, or headless scripts. A shared driver manages robot environments like LIBERO and LIBERO-Plus, supporting scene reset and switching via slash commands. The system also supports two backends: the original OpenPI VLA version and the newer Cosmos Policy WAM version.

Who it’s for

Researchers and developers working on embodied AI, robot learning, or long-horizon manipulation tasks. The project provides evaluation suites for standard LIBERO benchmarks and robustness tests, making it suitable for those who want to compare multi-policy collaboration against single-policy baselines. It requires comfort with Conda environments, model checkpoints, and running multiple GPU services.

Highlights

• Replaces single end-to-end policies with a graph of collaborating specialized models (planning, perception, verification, action)
• Supports two model backends: OpenPI VLA and Cosmos Policy WAM, both with isolated model servers and batched inference
• Offers three interaction modes: web console with live camera views, terminal UI with slash commands, and headless execution for scripts
• Includes evaluation support for LIBERO, LIBERO-Plus, and LIBERO-Pro benchmarks