Orkas-AI/Orkas-VideoStudio

What it solves

OrkasVideoStudio provides a way for AI coding agents (like Claude Code, Cursor, or Codex) to compose, generate, and edit videos using natural language. It moves away from "black-box" AI video generation by using a readable, editable, and diffable intermediate representation (plan.json), allowing both the agent and the user to precisely modify specific parts of a video without re-rendering the entire project.

How it works

The system acts as a toolkit that provides the knowledge and deterministic capabilities for an agent to follow. It operates across four "production lines":

• Compose: Creates motion graphics and explainers from scripts using HTML/CSS (via HyperFrames).
• Edit: Performs deterministic video editing tasks like trimming silence, removing filler words, mixing audio, and burning in subtitles using ffmpeg and whisper.cpp.
• Generate: Interfaces with external AI providers (BYO keys) for talking-head or cinematic footage.
• Auto: An end-to-end pipeline that coordinates the other three lines into a single plan.json and uses a "delivery guard" to verify the final output meets the original requirements.

Agents interact with the toolkit via a CLI (ovs) or an MCP server, using provided "skills" (knowledge packs) to understand how to route requests and execute the production pipeline.

Who it’s for

• Developers using AI coding agents who want to automate video production.
• Users who want an auditable, local-first video creation workflow where they retain control over their API keys and the final timeline.
• Teams needing to generate explainers, highlight clips from long recordings, or localized video content automatically.

Highlights

• Editable IR: Uses plan.json so changes to one line of a script or one clip only require a partial re-render.
• Agent-Agnostic: Works with any agent capable of running a shell or speaking MCP.
• Zero-Key Trunk: Core composition and editing features work without paid API keys.
• BYO Generation: Supports various providers for image, video, and TTS (e.g., OpenAI, Gemini, MuAPI) without vendor lock-in.
• Deterministic Guard: Includes a promise-check tool to verify the final video's duration, audio, and motion before delivery.