mexicat/pdoom-video

What it solves

It provides a system for creating a generative, code-rendered music video where every frame is a deterministic function of the song's time. This ensures that a real-time browser preview and a high-resolution offline export (up to 4K60) remain identical.

How it works

The project uses a combination of audio analysis and a TypeScript/three.js renderer:

• Audio Analysis: Python tools (using Demucs, Whisper, and CTC forced alignment) analyze the song to produce word-level lyric timings and audio data (BPM, beats, onsets, and loudness envelopes).
• Rendering Engine: A TypeScript-based engine built with three.js, Bun, and Vite renders scenes based on a timeline anchored to lyric lines and the beat grid. It includes post-processing effects like bloom, halation, and grain.
• Export Pipeline: A headless Chrome instance (via Playwright) renders frames, which are streamed via WebSocket to ffmpeg for final video encoding. It features a sophisticated motion blur system that adaptively samples sub-frames based on movement intensity.

Who it’s for

Digital artists, creative coders, and developers interested in generative art, synchronized audio-visualizations, and high-fidelity code-driven video production.

Highlights

• Deterministic Rendering: Guaranteed consistency between live browser previews and final 4K exports.
• Adaptive Motion Blur: A sampling system that adjusts sub-frame counts per 32x32 tile to optimize render speed while maintaining visual quality during fast motion.
• Automated Audio Sync: Integrated pipeline for stem separation and forced alignment to achieve precise word-synced karaoke typography.
• High-Resolution Support: Native 4K rendering where every layer and shader is rendered at physical resolution rather than upscaled.