via007/bilibili-rag

What it solves

It transforms Bilibili video collections (such as interviews, lectures, and courses) into a searchable, conversational personal knowledge base. This allows users to retrieve specific information and ask questions about the content of their saved videos without having to re-watch them entirely.

How it works

The system follows a pipeline to process video content into a queryable format:

1. Content Extraction: It logs into Bilibili to read the user's collections and pulls the video content.
2. Transcription: It uses Automatic Speech Recognition (ASR) to convert video audio into text. If direct audio links are unavailable, it downloads the audio locally using ffmpeg and uploads it for processing.
3. Indexing: The transcribed text is converted into vectors and stored in a vector database (ChromaDB) alongside a traditional database (SQLite).
4. Retrieval & Generation: Using a RAG (Retrieval-Augmented Generation) architecture, the system retrieves relevant text segments based on a user's query and uses an LLM to generate a grounded answer.

Who it’s for

• Students and researchers organizing technical videos or open courses.
• Professionals archiving podcasts, interviews, or meeting recordings.
• Anyone looking to summarize and review knowledge from Bilibili video collections.

Highlights

• Automated Pipeline: Handles everything from collection pulling to ASR and vector indexing.
• Multi-part Support: Capable of processing multi-part (multi-P) videos sequentially.
• RAG-based Chat: Provides conversational answers with traceable sources.
• Export Options: Allows exporting original content or AI-generated notes as Markdown.
• Integration: Includes a skill for OpenClaw to allow external agentic access to the knowledge base.