NoizAI/HelixWorld

🪐 HelixWorld: real-time interactive audio-visual world model.

What it solves

HelixWorld creates a real-time interactive audio-visual world model. It allows users to navigate a scene—walking forward or turning around—where both the visual and audio components update dynamically and synchronously based on the camera's movement and the user's prompts.

How it works

Starting with an image and a text prompt, the model generates a navigable environment. As the user provides movement actions (such as walking forward or turning), the system updates the spatial field of both the picture and the sound, ensuring the audio is not a soundtrack but a spatially aware sound field that follows the viewpoint.

Who it’s for

Developers and researchers interested in world models, interactive AI-generated environments, and synchronized audio-visual generation.

Highlights

  • Joint audio-video generation: Visuals and sound are born together and update in sync.
  • Camera navigation: Supports roaming via actions like W, A, S, D and turning.
  • Spatial sound field: Audio updates dynamically based on the camera's perspective.
  • Interactive demo: Available via a web browser at helixworld.org.
  • Offline inference: Preview v1 provides code and checkpoints for local execution on NVIDIA GPUs.

Related

  • Project
    Tencent-Hunyuan/HY-WorldPlay

    HY-World 1.5 is a real-time interactive world modeling framework that uses streaming video diffusion to generate geometrically consistent 3D environments based on user input.

  • Project
    SegFault42/HeliosGen

    HeliosGen is an open‑source desktop app (macOS released, Windows/Linux pending) that provides a visual, node‑based canvas for building and running AI image and video generation pipelines. It works locally, sending requests to the kie.ai API (or optionally a Codex CLI) and stores all data on the user’s machine. Features include multi‑model support, reference images, parallel/sequential execution, and a real‑time history. The app is built with Tauri 2 (Rust) and a Next.js/React front‑end, uses SQLite for local persistence, and is MIT‑licensed.

  • Project
    AMAP-ML/DreamX-World

    DreamX-World is a general-purpose interactive world model that generates high-fidelity, controllable simulations allowing users to explore and transform environments via event prompts.

  • Project
    fudan-generative-vision/Hallo-Live

    Hallo-Live is a real-time text-driven framework that generates synchronized audio and video for digital avatars using a causal dual-stream Diffusion Transformer.

  • Project
    YGYOOO/WorldX

    WorldX is an AI-powered simulation engine that generates a complete virtual world with autonomous agents, maps, and emergent narratives from a single text prompt.