Robbyant/lingbot-world-v2

🤖 What is LingBot‑World‑V2?

LingBot‑World‑V2 (also called LingBot‑World‑Infinity) is a research‑grade generative model that creates interactive, video‑level worlds from text prompts. It builds on the original LingBot‑World project and adds four major upgrades:

Unbounded interaction horizon – the model can keep generating coherent video frames for arbitrarily long scenes while preserving quality, thanks to a causal pre‑training scheme.
Real‑time speed – a distilled “fast” variant runs fast enough to drive 720p video at 60 fps on multi‑GPU rigs.
Richer interaction set – new actions such as attacking, archery, spell‑casting and shooting, plus many more text‑driven events, give the world a much broader behavioural vocabulary.
Agentic harness – a pilot agent plans character behavior while a director agent continuously creates novel environmental elements, enabling truly dynamic world evolution.

The repository ships the inference code (no training code) and links to the released model checkpoints.

📦 Quick start (for a non‑expert)

Clone & install

git clone https://github.com/robbyant/lingbot-world-v2.git
cd lingbot-world-v2
# PyTorch ≥2.4 is required
pip install -r requirements.txt
# Flash‑Attention speeds up the transformer kernels
pip install flash-attn --no-build-isolation

Download a model – pick one of the four released checkpoints. Example for the 14 B fast model (Hugging Face):
```
pip install "huggingface_hub[cli]"
huggingface-cli download robbyant/lingbot-world-v2-14b-causal-fast \
    --local-dir ./lingbot-world-v2-14b-causal-fast
```
(The 1.3 B model shares the T5/VAE tokenizer and VAE weights with the 14 B release; pass the 14 B folder via --assets_dir.)

Run inference – the provided generate.py streams video frames chunk‑by‑chunk, which keeps memory low.

# Example: 14 B fast model, 480p video, 8 GPUs
torchrun --nproc_per_node=8 generate.py \
    --task i2v-A14B \
    --size 480*832 \
    --ckpt_dir lingbot-world-v2-14b-causal-fast \
    --image examples/03/image.jpg \
    --action_path examples/03 \
    --dit_fsdp --t5_fsdp \
    --ulysses_size 8 \
    --frame_num 361 \
    --local_attn_size 18 --sink_size 6 \
    --prompt "A serene lakeside scene …"

For the smaller 1.3 B fast model replace --nproc_per_node=4 and add --assets_dir lingbot-world-v2-14b-causal-fast.

A helper script run_fast.sh can be used instead of writing the full command:

bash run_fast.sh lingbot-world-v2-14b-causal-fast 361

Watch the output – the script writes a video file (e.g., output.mp4) that you can play with any media player.

📂 What’s in the repo?

Folder / file	Purpose
`generate.py`	Main inference script (causal KV‑cache, chunked video generation).
`run_fast.sh`	Convenience wrapper that infers GPU count and model size from the checkpoint name.
`requirements.txt`	Python dependencies (torch, transformers, flash‑attention, etc.).
`examples/`	Sample images and action‑path files used in the demo prompts.
`README.md` (this file)	Documentation, model download table, quick‑start guide.

The code is built on Wan2.2 (a video diffusion framework). For full installation details of Wan2.2, see its own repository.

📦 Available model checkpoints

Model	Size	Type	Where to get
`lingbot-world-v2-14b-causal-fast`	14 B	Real‑time distilled (fast)	🤗 [HuggingFace] & 🤖 [ModelScope]
`lingbot-world-v2-14b-causal-pretrain`	14 B	Causal pre‑trained (high‑quality, slower)	🤗 [HuggingFace]
`lingbot-world-v2-14b-bid`	14 B	Bidirectional (research)	🤗 [HuggingFace]
`lingbot-world-v2-1.3b-causal-fast`	1.3 B	Light‑weight fast variant	🤗 [HuggingFace]

All models are released under CC BY‑NC‑SA 4.0 (non‑commercial use only). The 1.3 B package currently contains only the diffusion (DiT) weights; the tokenizer, T5 encoder, and VAE are shared with the 14 B release.

🌐 Try it online

International web demo – hosted on Reactor: https://www.reactor.inc/lingbot-world-v2
Domestic mobile demo – hosted on LingGuang: https://www.lingguang.com/support

Both platforms run the full‑capability model in real time. For the official demo at the WAIC 2026 conference, follow the link in the README.

📚 Related work & citations

Original LingBot‑World project: https://github.com/robbyant/lingbot-world
Technical report (arXiv): https://arxiv.org/abs/2607.07534

If you use the model in research, please cite:

@article{lingbot-world-v2,
  title={Infinite Worlds with Versatile Interactions},
  author={Zelin Gao and Qiuyu Wang and Jiapeng Zhu and Jingye Chen and Zichen Liu and Qingyan Bai and Jiahao Wang and Yufeng Yuan and Hanlin Wang and Yichong Lu and Ka Leong Cheng and Haojie Zhang and Jian Gao and Tianrui Feng and Yuzheng Liu and Yao Yao and Yinghao Xu and Xing Zhu and Yujun Shen and Hao Ouyang},
  journal={arXiv preprint arXiv:2607.07534},
  year={2026}
}

📄 License

The code and model weights are released under Creative Commons Attribution‑NonCommercial‑ShareAlike 4.0 International. You may share and adapt the material for non‑commercial purposes, provided you give appropriate credit and distribute derivatives under the same license.

TL;DR

What: A video‑generation model that can keep a virtual world evolving indefinitely, with fast (real‑time) and high‑quality variants.
Why it matters: Enables interactive AI‑driven simulations, games, or virtual‑environment research where the world can react to textual commands over long horizons.
How to start: Install dependencies, download a checkpoint, run generate.py (or run_fast.sh) with a prompt and a source image.
Where to see it: Live demos on Reactor (web) and LingGuang (mobile).