via007/bilibili-rag

What it solves

It transforms Bilibili video collections (such as interviews, lectures, and courses) into a searchable, conversational personal knowledge base. This allows users to retrieve specific information from their saved videos and ask questions based on the video content without having to re-watch the entire footage.

How it works

The system follows a pipeline to process video content into a queryable format:

1. Content Acquisition: It logs into Bilibili to read the user's collection folders.
2. Transcription: It extracts audio from videos and uses Automatic Speech Recognition (ASR) to convert speech to text. It includes a fallback mechanism to download audio locally via ffmpeg if direct links are inaccessible.
3. Indexing: The transcribed text is converted into vectors and stored in a vector database (ChromaDB) alongside a relational database (SQLite).
4. Retrieval & Generation: Using a RAG (Retrieval-Augmented Generation) architecture, the system performs semantic searches to find relevant text segments and uses an LLM to generate answers based on those segments.

Who it’s for

• Students and lifelong learners organizing technical videos or open courses.
• Professionals archiving podcasts, meetings, or sharing sessions.
• Anyone who wants to summarize and review knowledge from Bilibili video collections efficiently.

Highlights

• Automated Pipeline: Handles everything from collection pulling to ASR and vector indexing.
• Multi-part Support: Supports transcribing and indexing multi-part (multi-P) videos.
• Source Traceability: Answers are grounded in the transcribed video content.
• Export Options: Allows exporting original content or AI-generated notes as Markdown.
• OpenClaw Integration: Includes a skill for integrating the local service into the OpenClaw agent framework.