threerocks/sweety-handdraw-video

What it is

Sweety‑handdraw‑video is an open‑source “skill” that lets a language‑model‑driven Agent turn a piece of text (a topic, an article or a full story) into a hand‑drawn narrated video. The Agent writes a script, plans the storyboard and decides the visual style. Local Python tools then:

• draw each illustration stroke‑by‑stroke with Pillow/SciPy,
• render handwritten subtitles and on‑screen text,
• synchronise the drawing steps with a pre‑generated voice‑over, and
• assemble everything into an MP4 with FFmpeg.

The project ships a set of 21 predefined illustration styles, six hand‑arm presets, and six built‑in voice‑over profiles. Users can also supply their own assets.

---

Who it’s for

• Content creators who want short educational or storytelling videos where the audience sees the drawing process in real time.
• Teachers / trainers looking for a quick way to turn lecture notes into a visual explanation.
• Marketers or indie filmmakers who need a cheap, fully‑local pipeline for hand‑drawn explainer videos.

---

How it works (high‑level flow)

1. Prompt the Agent – send a command like 使用 $sweety-handdraw-video，把“为什么越生气，越容易做错决定”做成手绘讲解. The Agent:
   • extracts the key points,
   • writes a narration script,
   • creates a storyboard (scene order, timing),
   • picks a visual style, hand‑arm preset and voice profile.
2. Generate the voice – either:
   • run a local TTS model (VoxCPM2‑8bit on Apple‑silicon, or a self‑hosted VoiceStudio service), or
   • import an already‑recorded full‑length narration.
3. Draw each frame – Python scripts use Pillow to:
   • trace outlines, fill colours, and write the subtitles stroke‑by‑stroke,
   • animate the hand‑arm graphic according to the chosen preset.
4. Synchronise – timestamps from the narration drive when each drawing step starts/ends.
5. Export – FFmpeg combines the image sequence, audio and optional subtitles into a 1920×1080 @ 30 fps MP4.

All artefacts (script, timestamps, generated images, audio) are saved so you can edit a single part (e.g., change the watermark) without re‑rendering the whole video.

---

Key features

Feature	What it gives you
21 built‑in illustration styles	From minimalist black‑and‑white line art to watercolor‑like “暖光童画”. Styles are identified by a numeric ID (1‑20 + 3.1) and can be selected directly or let the model pick one based on content.
Six hand‑arm presets	Different hand‑arm drawings (e.g., navy, diamond-woman, pen-only) that appear while the picture is being drawn.
Six voice‑over profiles	Real‑recorded “逼真男声”, “逼真女声”, “甜美学姐”, etc., plus the ability to use a local TTS engine (VoxCPM2 or VoiceStudio) or an existing audio file.
Full‑pipeline automation	From raw text to final MP4 with one command, as long as the required local tools (Python 3.9+, Pillow, NumPy, SciPy, FFmpeg, Bun, ExifTool) are installed.
Editable output	Scripts, timestamps, subtitles, watermarks and pen‑rod text are stored separately, allowing targeted re‑renders.
Local‑only processing	No cloud video‑generation service is used; all drawing and audio synthesis happen on the user’s machine.
Skill‑based installation	Can be added via npx skills add threerocks/sweety-handdraw-video or by manually copying the folder into a Codex‑style Skills directory.

---

Getting started (quick checklist)

1. Install the skill – npx skills add threerocks/sweety-handdraw-video (or unzip the v1.2.3 release into ~/.codex/skills/).
2. Set up the Python environment:

   python3 -m venv .venv && source .venv/bin/activate
   pip install -r skills/sweety-handdraw-video/scripts/requirements.txt
   python skills/sweety-handdraw-video/scripts/validate_package.py
   

3. Make sure you have the external tools – FFmpeg/ffprobe, a Chinese font, Bun + ExifTool, and a TTS backend (VoxCPM2 on Apple‑silicon or a locally running VoiceStudio service).
4. Run a test prompt – send the example command shown in the README to your Agent (e.g., Codex). The Agent will reply with a plan and then invoke the local scripts to produce the video.
5. Find the results – the output folder contains video.mp4, a cover PNG, an SRT subtitle file, the generated script and a log of timestamps.

---

Limitations & open questions

• Platform support – The full end‑to‑end flow has only been verified on macOS/Linux with Apple‑silicon (M3 Max). Windows and other hardware may require extra work, especially for the VoxCPM2 TTS model.
• Cost of upstream models – The image‑generation and TTS models that the Agent calls may be commercial or require separate licences; the repository does not ship them.
• No background music – The pipeline deliberately outputs pure narration; adding music would need a manual post‑process step.
• Batch processing – The tool can be scripted for multiple inputs, but there is no built‑in queue manager; each video still goes through the full draw‑and‑sync cycle.
• Handwriting fidelity – Strokes follow a simple outline‑then‑fill algorithm; they do not guarantee correct Chinese stroke order or calligraphic style.

---

TL;DR

sweety-handdraw-video is a genuine, open‑source project that combines LLM‑driven planning with local Python rendering to produce hand‑drawn narrated videos from plain text. It ships ready‑made visual styles, hand‑arm presets and voice profiles, runs entirely on the user’s machine, and is aimed at creators who want quick, editable explainer videos without cloud services.