ysyecust/lecture-to-notes

What it solves

Watching a lecture video is passive and hard to review later. This project automatically turns a video URL into a structured, reader-friendly Chinese handout: a LaTeX source file, a compiled PDF, cropped figures, corrected subtitles, and verification reports. It reorganizes the content around reader questions instead of following the speaker's spoken flow, and every figure carries a footnote showing exactly which part of the video it came from, while every numerical claim is checked against the transcript or on-screen text.

How it works

The pipeline takes a video link from YouTube, Bilibili (including multi-part playlists), or X/Twitter. It first identifies the platform and fetches metadata. Then it tries to obtain subtitles using a five-level fallback: manual CC, platform auto-subtitles, OCR from burned-in hard subtitles, local automatic speech recognition (ASR), and finally dictionary- or LLM-based correction. In parallel, it downloads the video and extracts frames, then crops away static navigation bars and subtitle bands, and if the video shows a presenter camera plus a slide area, it keeps only the slide portion. After gathering subtitles and cleaned frames, a reader-first writing step generates the LaTeX notes: chapters are organized around reader questions, formulas get symbol tables, and numbers are logged in a separate table for cross-checking. The notes are compiled twice with xelatex, then verified by a gatekeeper script that checks for completeness, figure-time consistency, and whether figures stay on the same page as their footnotes. Only if all checks pass does the final PDF and all supporting artifacts get produced.

Who it's for

Anyone who wants high-quality, easily reviewable notes from online lectures—especially Chinese-speaking students or self-learners who prefer structured handouts over raw videos. It's also useful for educators who want to convert existing lecture recordings into polished course materials. The project provides a ready-to-use agent skill for Claude Code, Codex, or DeepSeek Harness, so users with those tools can trigger the whole process with a simple command. The included online course library, with over 100 lectures across 9 courses, shows what the output looks like and can be browsed in both web-optimized text and original PDF modes.

Highlights

• Five-level subtitle fallback – from manual CC to OCR to local ASR, ensuring captions are available even when the platform doesn't provide them.
• Reader-first writing – chapters are reorganized around reader questions, not spoken order, and formulas are always followed by symbol explanations.
• Time-traceable figures – every image in the notes has a footnote with the exact video time range, and a verification step guarantees the figure and its footnote stay on the same page.
• Numerical claim checking – numbers with units are extracted into a separate TSV file and cross-checked against the transcript.
• Dual-format reading – web pages with adjustable font size and a click-to-jump-to-video feature, plus an original PDF reader with full-text search and zoom.
• Agent skill integration – installable as a skill for Claude Code, Codex, and DeepSeek Harness; just type /lecture-to-notes <URL>.
• Active course library – 9 courses, 101 lectures, 2,451 pages of Chinese handouts already available online.