gnipbao/story-to-handdrawn-video

What it solves

This project automates the creation of vertical, hand-drawn animations from Chinese stories or a sequence of images. It eliminates the manual effort of storyboarding, illustration generation, and video editing by using an AI Agent to orchestrate the entire pipeline from text to final render.

How it works

The system splits the process into content generation and animation rendering:

1. Agent Orchestration: An AI Agent (via a provided "Skill") plans the storyboard, defines character consistency, and uses an image generation tool to create "master pages" containing both handwritten Chinese text and illustrations.
2. Asset Processing: Local scripts process these master pages. In "layered drawing" mode, the system derives a grayscale version from the color image to create a reveal effect. In "page-flip" mode, the original page is preserved.
3. Rendering: Remotion is used to animate these assets—either by sequentially revealing text, grayscale art, and color, or by simulating a page-curl effect from the bottom-right corner.
4. Output: The result is a silent 3:4 vertical MP4 video ready for post-production voiceovers or music.

Who it’s for

• Content creators making story-based short videos or digital picture books.
• Users of AI Agents (like Claude Code, Codex, or Kimi Code) who want to generate visual narratives from text.
• Artists and designers wanting to experiment with various hand-drawn styles using a large built-in library of 327 style assets.

Highlights

• Extensive Style Library: Includes 297 style recipes and 30 color palettes across 11 categories.
• Style Mimicry: Ability to analyze a reference image's lines, colors, and textures to generate new stories in a matching visual language.
• Integrated Text: Generates handwritten Chinese text directly into the artwork rather than overlaying digital fonts.
• Flexible Animation: Supports both a progressive "drawing" reveal and a realistic page-flip transition.