krillinai/KrillinAI

What it solves

KrillinAI is an end-to-end video localization and enhancement tool. It automates the tedious process of translating videos for different languages and platforms, handling everything from downloading the source video to generating dubbed audio and platform-specific layouts (landscape and portrait).

How it works

The tool implements a complete pipeline that can be operated via a desktop client, a web UI, or a command-line interface (CLI). It integrates several AI components:

• Transcription: Uses Whisper (OpenAI, FasterWhisper, WhisperKit, or WhisperCpp) or Alibaba Cloud ASR to convert speech to text.
• Translation: Employs LLMs (OpenAI, Gemini, DeepSeek, etc.) to translate subtitles while maintaining context and natural semantics.
• Dubbing: Uses TTS services (Alibaba Cloud, OpenAI, MiniMax) or CosyVoice for voice cloning and audio generation.
• Processing: Automatically handles subtitle alignment, terminology replacement, and video rendering for various social media formats (e.g., TikTok, YouTube, Bilibili).

Who it’s for

• Content Creators: People who want to localize their videos for global audiences with minimal manual effort.
• AI Agents: Developers building automated workflows that can orchestrate the translation and dubbing pipeline via the CLI and provided "Agent Skills."

Highlights

• Full-Lifecycle Pipeline: Covers download, transcription, translation, dubbing, and rendering in one tool.
• Agent-Ready: Provides a structured CLI and a dedicated skills/ directory for AI agents to invoke specific stages.
• Flexible Deployment: Supports Windows, Linux, and macOS with options for local model execution or cloud API usage.
• Cross-Platform Optimization: Automatically generates both horizontal and vertical video formats with optimized subtitle layouts.
• Voice Cloning: Supports custom voice cloning via Alibaba Cloud services.