Robbyant/lingbot-video

Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

What it solves

LingBot-Video is designed to bridge the gap between video synthesis and physical world understanding, specifically for embodied intelligence. It aims to generate high-quality videos that are physically rational and task-oriented, moving beyond simple aesthetic video generation to support the needs of robotics and embodied AI.

How it works

The project uses a large-scale Mixture-of-Experts (MoE) architecture, which allows for high model capacity while maintaining efficient inference (approximately 3x faster than dense models). It was trained on a massive dataset of web videos combined with over 70,000 hours of specialized embodied data. To ensure quality, it employs a multi-reward system that optimizes for aesthetics, physical rationality, and task completion.

Who it’s for

This tool is primarily for researchers and developers working in embodied intelligence, robotics, and video generation, as well as those looking for high-performance open-source video models that can handle complex physical interactions.

Highlights

  • MoE Architecture: Scaled from scratch to balance capacity and cost.
  • Embodied Data Engine: Integrated 70,000+ hours of embodied-specific video data.
  • Multi-Task Support: Supports Text-to-Image (T2I), Text-to-Video (T2V), and Text-to-Image-to-Video (TI2V).
  • Structured Prompting: Uses a dedicated prompt rewriter (based on Qwen3.6) to convert natural language into structured JSON captions for better control.
  • High Performance: Ranks top on the RBench leaderboard for embodied video generation.

Related

  • Project
    inclusionAI/Ming

    Ming-flash-omni 2.0 is an open-source omni-MLLM that unifies multimodal perception and generation across text, image, audio, and video in a single MoE-based architecture.

  • Project
    HITsz-TMG/Uni-MoE

    Uni-MoE is a Mixture-of-Experts based omnimodal large model capable of understanding and generating across multiple modalities, including text, images, and speech.

  • Project
    PaddlePaddle/ERNIE

    A family of large-scale multimodal MoE models and development toolkits providing state-of-the-art performance in text and vision-language understanding.

  • Dispatch
    Qwen-RobotWorld: Boundless Worlds for Embodied Agents

    Qwen-RobotWorld is a unified world model that uses natural language as a universal action interface to enable cross-scenario physical generalization across 20+ robot embodiments.

  • Project
    Robbyant/lingbot-world

    An open-source world simulator that generates high-fidelity, interactive video environments with real-time interactivity and long-term temporal consistency.