air-embodied-brain/Zetta-Embodiment

Zetta is an efficient closed-loop embodied harness for self-evolving physical intelligence. It evolves code-based runtime critics and recovery skills online while keeping the base policy frozen. Project Page: https://air-embodied-brain.github.io/zetta/

What it solves

Zetta is designed to improve the success rates of embodied AI agents (robots) by allowing them to self-evolve their behavior without needing to retrain the base model. It addresses the problem of failure recovery in complex physical tasks by creating a system that can diagnose why a robot failed and automatically generate code-based recovery skills.

How it works

Zetta uses a closed-loop "evolution protocol" to improve performance. Instead of updating the base policy (which remains frozen), it evolves "runtime critics" and "recovery skills" through a following process:

  1. Failure Clustering: Groups failed attempts into clusters based on video and telemetry data.
  2. Causal Diagnosis: A diagnostic stage identifies the specific cause of a failure.
  3. Candidate Generation: A system generates a bundle of a critic (to detect the failure) and a recovery action (to execute the code-based fix).
  4. Verification: The new skill is tested via "shadow replay" and gated checks against specific seeds to ensure it actually solves the problem without causing regressions.
  5. Promotion: Once verified, the skill is promoted to the active runtime.

Who it’s for

This system is built for researchers and developers working on Vision-Language-Action (VLA) models and robotic manipulation in simulated environments like LIBERO, RoboCasa, ManiSkill, and BEHAVIOR.

Highlights

  • High Success Rates: Achieves state-of-the-art results on LIBERO-Pro (90.8%) and RoboCasa (93.6%).
  • Inference Speedup: Provides an 11.1x inference speedup over traditional methods.
  • Broad Environment Support: Integrates with multiple simulators including Isaac Sim, OmniGibson, and SAPIEN.
  • Modular Runtime: Uses a separate Rollout Runtime to decouple simulator workers from model inference workers.

Related

  • Project
    OpenBMB/DeepThinkVLA

    DeepThinkVLA is a Vision‑Language‑Action model that inserts an explicit chain‑of‑thought reasoning step before generating robot actions. Using a hybrid decoder (autoregressive reasoning → bidirectional parallel action generation), the authors train on a newly curated embodied CoT dataset and then refine with outcome‑driven reinforcement learning. The model achieves 97 % average success on the LIBERO benchmark, runs at only 0.175× the latency of a pure autoregressive baseline, and transfers zero‑shot to the newer LIBERO Plus suite.

  • Project
    starVLA/starVLA

    StarVLA is a modular, open-source codebase for developing Vision-Language-Action models for robots. It offers plug-and-play components, multiple VLA frameworks, training recipes, and benchmark integrations, with a focus on rapid prototyping and easy customization.

  • Project
    gepa-ai/gepa

    GEPA is a framework that optimizes any text-controlled system (prompts, code, agent architectures, configs) against any measurable metric. It uses LLMs to read execution traces, diagnose failures, and iteratively evolve better candidates via reflection, mutation, and Pareto-aware selection — requiring far fewer evaluations than RL. It offers a simple `gepa.optimize()` API, an `optimize_anything` API for arbitrary text artifacts, DSPy integration, built-in adapters for common use cases, and integrations with MLflow, Comet, Pydantic, Google ADK, and more. Reported results include 90x cost reduction vs. Claude Opus 4.1 at Databricks, 35x faster than RL, and 50+ production uses.

  • Project
    bingreeky/MemGen

    MemGen is a research implementation of a latent‑memory mechanism for LLMs, featuring a Trigger that decides when to recall memory and a Weaver that creates compact memory tokens. The repo provides conda‑based setup, training/evaluation scripts for Qwen2.5‑1.5B‑Instruct and SmolLM3‑3B on KodCode, GSM8K, and TriviaQA, and includes ready checkpoints on HuggingFace.