myAiVideos – Automated Chinese‑language short‑video production pipeline

One command in, a publish‑ready video out. The repository implements a full end‑to‑end workflow that discovers a topic, writes a script, generates visuals and audio, does post‑production, and finally hands the clip to a distribution channel. All steps are orchestrated by a FastAPI service and a Java Spring‑Boot gateway that provide authentication, routing, circuit‑breaking and observability.

---

What the project does

• Topic discovery (L1) – pulls hot‑news or trending items and ranks them.
• Creative generation (L2) – builds a script/prompt, anchoring the visual identity of any on‑screen characters.
• Visual creation (L3) – uses text‑to‑image or image‑to‑video models (DeepSeek, Kling, MuAPI, etc.) and runs CLIP‑based consistency checks.
• Audio creation (L4) – synthesises speech with Volcengine or MiniMax TTS, mixes with background music, and aligns duration.
• Post‑production (L5) – automatic AV‑sync rescue, caption generation, cover image creation.
• Distribution (L6) – packages the final MP4 and returns it to the caller (or pushes to a bot).
• Optimization (L7) – scores the result, stores feedback for future runs.

The pipeline is layered and decoupled; each stage can be swapped for a different model or service without touching the others.

---

Core architecture

• FastAPI (Python) – the orchestration engine. Handles incoming webhook requests, stores job state in PostgreSQL, queues work in Redis, and runs background workers via ARQ.
• Java Spring‑Boot gateway – sits in front of the Python API. It validates JWTs, propagates a trace_id for cross‑language observability, applies Resilience4j circuit‑breakers, caches results with Caffeine, and records token usage via an AOP aspect.
• Observability – Langfuse captures the full call tree; Prometheus scrapes metrics from the Java gateway.
• Data stores – PostgreSQL for persistent metadata, Redis for queues, MinIO for generated media assets.
• Containers – everything is packaged in Docker Compose (orchestrator, worker, gateway, DB, Redis, MinIO).

---

Key features (as described in the README)

• Seven‑layer pipeline that can be independently replaced.
• Multi‑model failover with automatic rotation when a provider (DeepSeek, Qwen, GLM, etc.) is unavailable.
• AI consistency governance:
  • Prompt anchoring to keep characters visually consistent.
  • CLIP‑based text‑image gating to discard off‑prompt frames early.
  • Four‑tier audio‑video sync rescue (tempo adjustment, video padding, narration rewrite).
• Full‑stack observability – trace_id flows from Java to Python; Langfuse shows the call tree.
• Metering & rate‑limiting – token usage captured without code changes, aggregated per tenant.
• Production‑ready UX – Server‑Sent Events stream progress, single‑segment regeneration, rolling log archives.

---

Tech stack snapshot

Layer	Tech
Orchestration (Python)	FastAPI, asyncio, Pydantic, SQLAlchemy (asyncpg), Alembic, ARQ
Gateway (Java)	Spring Boot 3.5, WebClient, Resilience4j, Caffeine, Prometheus
Frontend	Vue 3, Vite, Vue Flow, Pinia, Tailwind
Data	PostgreSQL, Redis, MinIO
Observability	Langfuse, Prometheus
AI models	DeepSeek, Zhipu GLM‑4V, Kling v2.5, Volcengine / MiniMax TTS, faster‑whisper
Containerisation	Docker Compose

---

Getting started (quick‑start)

# clone and copy env template
git clone <your-fork>.git && cd myAiVideos
cp .env.example .env   # fill in API keys for DeepSeek, GLM, Kling, TTS, etc.

# start core services (Postgres, Redis, FastAPI orchestrator, ARQ worker)
docker compose up -d postgres redis orchestrator worker

# apply DB migrations
docker compose exec -T orchestrator alembic upgrade head

# (optional) start the Java gateway
docker compose up -d gateway

# launch the Vue frontend
cd sl-vue && npm install && npm run dev

Health endpoints:

• FastAPI: http://localhost:8000/health
• Java gateway: http://localhost:8080/actuator/health
• Frontend: http://localhost:5173

---

Who might use this

• Content creators who need to produce Chinese short‑form videos (news commentary, explainers, storytelling, fact‑snippets) at scale.
• Product teams building AI‑driven video bots for platforms like Douyin, Kuaishou, or corporate internal feeds.
• Researchers / developers experimenting with multi‑modal generation pipelines, failover strategies, or AI governance (prompt anchoring, CLIP gating).

---

Documentation & further reading

• architecture.md – detailed data‑flow diagram and component responsibilities.
• docs/operations-manual.md – how to integrate with Feishu, run daily maintenance, and monitor the system.
• gateway/README.md – Java gateway configuration, auth, and metering details.
• Test suite (tests/) can be run via Docker Compose.

---

License

MIT – see LICENSE.

---

Bottom line: myAiVideos is a production‑grade, open‑source framework that stitches together several LLMs, text‑to‑image/video models, and TTS services into a robust, observable pipeline for automatically generating short Chinese videos. It is a genuine AI‑focused software project, not a mere tutorial or link collection.