Tencent-Hunyuan/Hy-Nomas

Hy-Nomas: Free-Form Interactive World Generation

Hy-Nomas is an open-source model from Tencent that generates interactive video worlds. You control it with text prompts and camera movements, and it creates video segments that respond to your inputs. It's designed to run on a single high-end GPU (like an RTX 4090) and includes tools for fine-tuning and an interactive web demo.

What does it do?

Generates video worlds: It creates video segments (e.g., 5-second clips) that form a continuous, explorable world.
Multi-turn control: You can give text instructions or camera moves at any time, and the model updates future segments accordingly.
Free-form prompts: You can describe objects, styles, scenes, or actions in natural language—no fixed templates needed.
Camera control: You can specify camera trajectories (like pan, zoom, or WASD-style movement) to explore the world.

Key features (from the README)

Segment-based generation: The model generates video in segments, one after another, using a "lifelong KV cache" to remember what came before. This keeps the world consistent across segments.
Distilled inference: A faster version of the model that uses only 4–8 denoising steps (instead of many) and a specialized decoder, making it practical for real-time use.
Low-memory inference: It can run on a single RTX 4090 with reduced memory usage (INT8 weights, CPU offloading, etc.).
LoRA fine-tuning: You can adapt the model to specific tasks (e.g., cinematic style, robotics) by training small adapter weights and merging them into the main model.
Interactive demo: A web interface that lets you control generation across multiple turns and see results at segment boundaries.

How to use it

Setup: You can use a pre-packaged Docker image (recommended) or install manually with Python 3.12+, uv, and CUDA 13.0.
Model assets: Download all weights from Hugging Face (tencent/Hy-Nomas). The repository includes different checkpoints (base, distilled, quantized) and adapters (cinematic, robotics).
Inference: Run scripts from the repo, or use the Diffusers pipeline (HyNomasModularPipeline) for Python integration.
Interactive demo: Either use the hosted online demo or run your own with the Reactor runtime or the web demo deployment guide.

Community integrations

Several third-party projects have added Hy-Nomas support:

SGLang: For offline video jobs and real-time streaming.
ComfyUI: A node-based interface with chained segments and camera control.
Reactor Runtime: A streaming service with live prompt updates and camera control.
WanGP: Adds keyboard exploration and text-driven events.
Google TPU demos: Runs on TPUs with real-time 720p generation.

Fine-tuning

You can fine-tune Hy-Nomas on your own video data:

Prepare data: Use the provided captioning harness to annotate videos and encode them.
Pack sequences: Build a packing plan for efficient training.
Train: Use the provided config to train LoRA adapters on a non-distilled checkpoint.
Merge: Merge the trained LoRA weights into the distilled checkpoint for inference.

License

Apache 2.0 (open source).

For full details, see the README.