SAOS-3624/dsh-video-storyboard

Scene-aware storyboard for DeepSeek Harness: 9-16 numbered cells with timestamps + transcript + an audio envelope, packed into ONE image (384 tokens). Two tools: video_storyboard and video_zoom.

What it solves

This tool solves the problem of high token costs and information loss when using AI to analyze videos. Instead of sending hundreds of individual frames (which consumes too many tokens) or relying solely on subtitles (which misses visual context), it creates a concise, multi-modal summary of a video.

How it works

It processes a video into a single "storyboard" image (a grid of key frames) paired with a time-coded text list. This list includes timestamps, dialogue, and shot counts. By condensing the visual information into a small, token-efficient format, it allows AI models to understand the video's content, pacing, and dialogue without needing to process every single frame.

Who it’s for

It is designed for users who want to quickly understand what a video is about, identify key moments, or locate specific scenes without the high cost of full-frame analysis. It is ideal for those who need a balance between visual context and textual detail.

Highlights

  • Token-Efficient: Summarizes 90 seconds of video into roughly 1,900 tokens, significantly cheaper than frame-by-frame analysis.
  • Multi-Modal Summary: Combines a visual grid of frames with a text-based transcript and shot-list.
  • Flexible Analysis: Supports local video files and links via yt-dlp, with optional ASR (Automatic Speech Recognition) for dialogue.
  • Deep Zoom: Includes a video_zoom feature to analyze specific segments in higher detail if the initial summary is insufficient.
  • Evidence-Based: Generates an .evidence.json file for every storyboard, allowing users to verify timestamps and content against the original video.

Related

  • Project
    a312863063/Video-Auto-Wipe

    A deep learning video inpainting project that automatically detects and erases fixed-pattern content like subtitles, static logos, and dynamic icons from videos. It uses a two-step training process (time-domain perception + end-to-end inpainting fine-tuning) with synthetic foreground data overlaid on real video backgrounds. The subtitle removal model is open-sourced, while logo removal models are demonstrated but not released.

  • Project
    QuanjianSong/FashionChameleon

    FashionChameleon is a real-time framework for interactive human-garment customization in video generation, enabling high-speed garment switching at 23.8 FPS.

  • Project
    Bomx/super-video-maker-skill

    An end-to-end AI video production skill for agentic coding frameworks that enables AI agents to create polished videos using avatars, b-roll, and automated composition.

  • Project
    minnie-lin/Awesome-Physics-Cognition-based-Video-Generation

    This repository organizes research papers, code, and resources on physics-cognition-based video generation. It provides a structured taxonomy, a survey paper, and links to recent work, helping researchers track the field.