myccarl/ai-shortVideo-pipeline

End-to-end AI short-video production pipeline. FastAPI orchestration + Spring Boot gateway with multi-model failover, circuit breaker, metering, and full-stack observability. AI quality gating: prompt anchoring, CLIP consistency, AV sync auto-rescue.

What it solves

It automates the entire process of creating short-form videos, transforming a single command into a publish-ready video. It specifically targets content types like news commentary, knowledge explainers, and storytelling, removing the the manual effort required to coordinate multiple AI tools for scripting, visuals, and audio.

How it works

The system uses a seven-layer decoupled pipeline architecture orchestrated by FastAPI and a Java Spring Boot gateway. The process flows through these stages:

  1. Topic Discovery (L1): Finds and ranks trending topics.
  2. Creative Generation (L2): Generates scripts and prompts.
  3. Visuals (L3): Produces images and videos using models like Kling AI, with CLIP-based consistency gating to ensure visuals match the prompt.
  4. Audio (L4): Handles text-to-speech (TTS) and mixing.
  5. Post-production (L5): Manages audio-video synchronization and captioning.
  6. Distribution (L6): Delivers the final product to the client.
  7. Optimization (L7): Scores quality and collects feedback.

To ensure reliability, it includes a Java gateway for multi-model failover, circuit breakers, and rate limiting, while using Langfuse for full-stack observability.

Who it’s for

Content creators and developers who want to automate the production of AI-generated short videos at scale, particularly for Chinese-language content.

Highlights

  • Seven-layer architecture: Decoupled stages from topic discovery to distribution.
  • AI Consistency Governance: Uses prompt anchoring and CLIP gating to maintain visual identity and reject off-prompt frames.
  • Multi-model Failover: Automatically rotates between providers (DeepSeek, Qwen, GLM) via a Java gateway to prevent downtime.
  • Production-ready Tooling: Includes SSE progress streaming, full-stack tracing with trace_id, and tenant-based metering.

Related

  • Project
    xhongc/ai_story

    An AI-driven automation platform that transforms a story theme into a complete video by automating scriptwriting, storyboarding, image generation, and video production.

  • Project
    pexoai/pexo-skills

    A set of open-source agent skills that automate the full video production pipeline, transforming text, images, or URLs into finished multi-shot videos with music and subtitles.

  • Project
    Anil-matcha/Open-AI-Micro-Drama-Generator

    An agentic AI pipeline that transforms a simple idea or script into a complete cinematic micro-drama video by automating story development, character design, and video production.

  • Project
    xuanyustudio/LocalMiniDrama

    An open-source, local-first AI short drama creator that automates the entire production pipeline from script generation to final video synthesis using various AI APIs.

  • Project
    YILS-LIN/short-video-factory

    An AI-powered desktop application that automates the creation of marketing and general short videos by combining AI copywriting, speech synthesis, and automatic video editing.