taoufik123-collab/claude-watch

What it solves

Claude cannot natively "watch" videos. This project provides a skill that allows Claude to process video files or URLs (YouTube, TikTok, etc.) by extracting key frames and transcripts, enabling the AI to answer questions about visual and audio content without the user having to watch the entire video.

How it works

1. Processing Pipeline: The tool uses yt-dlp to download videos and ffmpeg to extract frames. It employs an auto-scaling frame budget (capped at 100 frames) to manage token costs while maintaining visual coverage.
2. Transcription: It first attempts to pull native captions; if unavailable, it uses Whisper (via Groq or OpenAI) to generate a timestamped transcript.
3. Multimodal Input: The extracted frames (as JPEGs) and the transcript are fed into Claude's context, allowing the AI to "see" and "hear" the content.
4. Specialized Analysis: It includes a "hook microscope" for dense analysis of the first 10 seconds and scene-change detection to capture one frame per cut rather than at fixed intervals.
5. Integration: It can be installed as a plugin for Claude Code, a skill for claude.ai, or a Codex skill.

Who it’s for

• Content Creators: To analyze viral hooks and competitor content structures.
• Developers: To diagnose bugs from screen recordings.
• Researchers: To summarize long videos quickly without watching them at 2x speed.

Highlights

• Scene-Change Extraction: Captures frames based on visual cuts to keep token costs flat regardless of video length.
• Hook Microscope: Provides dense frames and word-level transcription for the first 10 seconds of a video.
• Obsidian Integration: Optional auto-save functionality to turn watched videos into connected wiki entries.
• Focused Mode: Supports --start and --end flags to perform a denser scan of specific time windows.