EternalEvan/Astra

What it solves

Astra addresses the challenge of creating realistic, long-horizon video predictions that respond to specific action inputs. It aims to provide a controllable and temporally coherent way to simulate future video sequences across diverse real-world scenarios, such as autonomous driving, robotic manipulation, and camera motion.

How it works

Astra is an interactive world model built on an autoregressive diffusion transformer with temporal causal attention. It enables streaming predictions by predicting future frames based on history and action signals. To maintain stability and control, it employs three key technical components:

• Noise-augmented history memory: Stabilizes the generation of long video rollouts.
• Action-aware adapter: Ensures precise integration of control signals.
• Mixture of action experts: Routes different types of action modalities to the appropriate processing paths.

Who it’s for

Researchers and developers working on autonomous systems, robotics, and AI-driven video generation who need a high-fidelity, action-conditioned simulator for predicting future visual states.

Highlights

• Action-Driven Control: Supports various action inputs (e.g., specific camera trajectories like S-shaped or rotating) to guide video generation.
• Long-Horizon Rollouts: Designed for streaming prediction with strong temporal coherence over extended sequences.
• Broad Application Range: Applicable to autonomous driving, drone navigation, and robot manipulation.
• Efficient Inference: Can run on a single 24GB GPU (e.g., NVIDIA 3090).