kajisho5/ffmpeg-skill

What is ffmpeg‑skill?

ffmpeg‑skill is a local, agent‑friendly wrapper around FFmpeg that lets AI coding agents (Claude Code, Cursor, Codex, or any MCP client) edit video and audio files without ever sending the media to the cloud. It ships a collection of 42 small Python tools (e.g., cut.py, fit.py, loudness.py, caption.py) that each expose a typed command‑line interface and a machine‑readable contract describing their inputs, outputs, and required FFmpeg capabilities.

The skill follows a strict workflow:

1. Probe the source with probe.py to get exact metadata (duration, fps, resolution, colour space, audio layout, HDR, etc.).
2. Edit losslessly when possible (stream‑copy) and only re‑encode when a filter or precise cut is needed.
3. Verify the result by probing it again and optionally generating a PNG contact sheet (look.py).
4. Return a structured JSON report that the calling agent can parse.

All of this runs offline, requires only FFmpeg, ffprobe, and Python 3 (standard library only). No API keys, no external services, and optional speech‑to‑text works only if you have a local ASR engine installed.

---

Key Features

Feature	Why it matters
Agent‑ready contract (contract --json)	Agents can read a JSON schema that guarantees the flags they pass match the underlying argparse definitions – no drift between docs and code.
42 focused tools	Each operation (cut, join, silence removal, captioning, HDR→SDR, loudness normalisation, etc.) is a separate script with typed arguments, making it easy for an LLM to invoke the right one.
Lossless‑first workflow	Tools try stream‑copy (-c copy) whenever the edit does not require re‑encoding, preserving quality and saving time.
Dry‑run & planning (--dry-run, --plan FILE)	Agents can preview the exact FFmpeg command lines before any file is written, or generate a reproducible plan that can be executed later.
Verification step (look.py, check.py)	After each edit the output is probed again; if the visual content changed a PNG sheet is produced so the agent can “see” what happened.
Capability detection (doctor)	Before running a job the skill checks whether the local FFmpeg build contains the required encoders/filters (e.g., libass, zscale, loudnorm).
No external dependencies	Pure Python 3.9+ standard library; optional local ASR engines (Parakeet‑mlx, faster‑whisper, openai‑whisper) are only used if present.
Template‑driven delivery (render.py --template tiktok)	Pre‑built JSON templates for platforms like TikTok, Reels, Shorts, etc., automatically set aspect ratio, caption style, loudness target, and export presets.
Cross‑repo ecosystem	Designed to be the “hands” of a larger AI‑video‑production OS; other skills (media‑analysis, transcription, subtitle, colour‑grading) can call its contract but not the other way around.

---

How to Get Started

# Install the skill (adds it to Claude’s skill directory)
npx ffmpeg-skill

# Verify your machine has the needed FFmpeg components
npx ffmpeg-skill doctor

# Inspect the JSON contract (useful for agents or custom tooling)
npx ffmpeg-skill contract --json | head -40

From the command line you can also run any tool directly, for example:

python3 ~/.claude/skills/ffmpeg-skill/scripts/cut.py input.mp4 \
    --segments 0:45-3:10,5:00-6:30 --dry-run

Or render a full platform‑specific package:

python3 $HOME/.claude/skills/ffmpeg-skill/scripts/render.py talk.mp4 \
    --template tiktok --cues cues.txt

---

Typical Use‑Case Flow for an AI Agent

1. User request – “Make a 60‑second Reel from interview.mp4 using the best highlights.”
2. Agent reads SKILL.md to understand the workflow.
3. Probe – probe.py returns exact metadata.
4. Decide cuts – agent may call scenes.py for highlight proposals, then cut.py with the chosen segments.
5. Fit & format – fit.py to enforce 9:16 aspect, export.py --preset reels for the correct codec/bitrate.
6. Check – check.py --platform reels validates duration, loudness, etc.
7. Look – look.py produces a contact sheet; the agent inspects the PNG to confirm the visual result.
8. Report – the skill returns a JSON object containing status, output path, probe summary, and a Look: line pointing to the PNG.

---

Who Might Use It?

• Developers building AI‑driven video editors that need a reliable, offline FFmpeg backend.
• Prompt engineers who want their LLM agents to manipulate media without writing raw shell strings.
• Content creators who prefer a local, privacy‑first workflow for quick platform‑specific cuts.
• Researchers experimenting with multimodal agents that need to see the result of a video edit.

---

Limitations

• The skill does not decide what to cut, which captions to write, or which highlight is “interesting” – that logic belongs to the calling AI brain.
• It relies on a local FFmpeg build; missing filters will cause the corresponding tool to be unusable (reported by doctor).
• Audio transcription is optional and only works if a supported local ASR engine is installed.

---

License

MIT – free to use, modify, and embed in other projects.