Endless1936/book-video

What it solves

This project provides an automated workflow for creating short promotional videos for books using natural language commands. It streamlines the process of book selection, copywriting, visual generation, and audio synchronization to produce a final video product.

How it works

Users interact with an AI agent (Codex) via natural language to guide the video production process. The workflow follows these steps:

1. Book Selection: The agent recommends books or confirms a specific title.
2. Copywriting: The agent generates a voice-over script that can be used in CapCut (剪映) or saved locally.
3. Visuals: Once the script is approved, the agent generates atmospheric images and video frames.
4. Audio: The agent uses voice cloning (if supported) or imports an MP3 file from CapCut.
5. Subtitles: It uses FFmpeg to generate subtitle timing references and uses the script as the source of truth for text, with Whisper used only for checking TTS content.
6. Iterative Refinement: Users can request changes to text, images, or audio, and the agent will regenerate the specific parts.

The system maintains a state file (workflow-state.json) to track the progress and dependencies of each episode, allowing for recovery and repair of the process if errors occur.

Who it’s for

Content creators and book promoters who want to produce high-quality short videos without needing to write code or manually handle complex editing software.

Highlights

• Natural Language Interface: Entirely controllable via conversational AI.
• Robust Error Handling: Includes a diagnostic system (BOOK_VIDEO_DIAGNOSTIC) to prevent bad outputs from replacing valid videos.
• State Management: Tracks production status per book to allow for easy reconstruction or repair of assets.
• Integrated Tooling: Combines image generation, TTS, FFmpeg, and Whisper for a complete production pipeline.