ziplab/ATAS

ATAS – Any Time, Any Space

What it is

ATAS is a research framework for agentic video generation that keeps a persistent world state across an entire film. A screenplay is compiled into a sequence of shot requests; each request pulls static geometry and dynamic event information from a shared world memory, conditions a video generator, and then the resulting clip is validated and written back into the world. This lets the system re‑film past moments from new viewpoints and ensures consistency throughout the video.

Core ideas

Component	Role
World‑State Compiler	Turns a plain‑text screenplay into bounded shot requests with explicit scene, camera, time, and event specifications.
Scene Memory	Stores static geometry of every location in a common coordinate frame, enabling rendering from any viewpoint.
Event Memory	Indexes dynamic observations (e.g., objects, actions) by scene, event, and time so past moments can be revisited.
Validated Updates	After a shot is generated, it is reconstructed, registered, and checked before the world state is updated, guaranteeing consistency.

How to get it running

1. Geometry service (GPU machine) – installs the heavy models (Pi3X and SAM‑3) and runs a REST service that renders the world.

   pip install -r geometry/environment.snapshot.txt && pip install -e geometry
   cp geometry/.env.example geometry/.env   # edit paths, GPU IDs
   cd geometry && ./run_geometry_service.sh   # should respond at http://127.0.0.1:18300/readyz
   

2. Client (any machine) – lightweight, only needs Python 3.12 and ffmpeg.

   pip install -e client
   cp client/.env.example client/.env   # set your DashScope API key, LLM settings, and the geometry service URL
   

Running a generation

atas run \
  --screenplay examples/bookstore/screenplay.txt \
  --characters examples/bookstore/characters.json \
  --story-id bookstore

• You can omit --characters (optional).
• Flags like --memory none, --plan-only, --max-shots N, or --output DIR give extra control.
• The command is resumable – re‑running it will reuse any already‑paid video generations.
• Output lives under client/runs/<run-id>/story_<story-id>/ and includes:
  • final.mp4 – the assembled film
  • manifest.json – per‑shot metadata
  • report.md – a short summary of the run

Where to look for more

• Design notes – docs/design.md
• Examples – examples/ (bookstore, clockshop, record)
• Tests – client && python -m pytest and geometry && python -m pytest
• Paper & video – arXiv link and a YouTube promo are listed in the badge bar.

License

MIT License for the ATAS code. Third‑party components under geometry/backbones/ retain their original licenses.

---

ATAS is a frontier‑research prototype; it requires a GPU for the geometry service and an external LLM (via DashScope) for the language‑to‑video conditioning.