YouTube Guitar Tab Parser: Automating Tablature Extraction from Video

YouTube Guitar Tab Parser: Automating Tablature Extraction from Video

YouTube Guitar Tab Parser is a CLI tool designed to automate the extraction of guitar tablature from YouTube lesson videos. By combining video processing tools and Large Language Model (LLM) vision capabilities, the tool converts a video stream into a structured PDF document containing the distinct lines of a song's tab.

Technical Architecture and Workflow

The tool follows a multi-stage pipeline to transform raw video into a final PDF. It leverages yt-dlp for video acquisition and ffmpeg for frame extraction, ensuring that only the necessary visual data is processed.

1. Video Acquisition and Frame Extraction

The process begins with yt-dlp fetching the video, with a resolution cap (default 720p) to optimize processing speed and API costs. Once downloaded, ffmpeg extracts one frame every two seconds by default. This sampling rate is designed to capture all changes in the tab lines as the video progresses.

2. Tab Region Detection

To avoid the high cost and inaccuracy of asking a vision model for precise pixel coordinates, the tool employs a labeled horizontal-band grid. A grid is drawn over a sample of frames (default 6 frames), and Claude vision is asked to identify which bands contain sheet music. The median first and last music band identifies the vertical extent of the tab region. This approach makes the region detection robust to outliers and prevents the from needing to precise coordinates.

3. Processing and De-duplication

Once the region is identified, the sharp library is used to crop every frame to that specific vertical area. To manage API costs and reduce the number of vision calls, the tool implements a two-tier de-duplication process:

  • Perceptual Hashing (dHash): A dHash perceptual hash is used to drop near-identical consecutive crops. This removes redundant frames where the tab has not changed.

  • Bar-number De-duplication: Claude vision is used to read the measure/bar number printed at the start of each line. Because the bar number remains constant while a playback cursor moves across a line, the tool keeps only the first appearance of a distinct bar number. This effectively collapses multiple frames of a single line into one single image.

4. PDF Generation

The final stage uses pdf-lib to stack the distinct tab lines vertically on A4 pages. The video title is automatically extracted from yt-dlp and used as the document metadata and a heading on the first page. The resulting file is saved to out/<video-title>.pdf.

Installation and Usage

To use the tool, users must have Node.js (version 20 or higher), yt-dlp, and ffmpeg installed on their system path. An Anthropic API key is required for the vision processing stages.

# Setup
npm install
npm run build
cp .env.example .env # Add your ANTHROPIC_API_KEY

# Execution
node --env-file=.env dist/cli.js "https://www.youtube.com/watch?v=WgU5tDGC-Vc"

Community Discussion and Considerations

The release of the tool has created a discussion regarding the ethics of extracting tabs from videos. Some users have argued that this tool facilitates a form of theft from creators who charge for PDFs of their tabs.

"Almost all those people charge for the PDFs. They know the PDF is more useful, that's why the video is free. I don't think your software is (or should be) illegal. But it's a form of theft, and incredibly unethical."

Conversely, other users have expressed relief at the tool's automating a tedious manual process of pausing, screenshotting, and assembling PDFs manually.

Technical Limitations

Community members have noted potential limitations in the tool's current approach:

  • Moving Tabs: The current logic may not work for videos where the tab moves (scrolling) rather than jumping to new lines.
  • Cost: The use of Claude vision for every distinct bar number can be costly compared to traditional computer vision (CV) techniques.
  • Curation: Some users noted that the only way to ensure accuracy is by purchasing official printed scores, as YouTube arrangements can sometimes be unique or incorrect.

Summary of Options

Option Default Description
-i, --interval 2 seconds Screenshot interval
--model claude-sonnet-5 Claude vision model used
--sample 6 frames Frames sampled for region detection
--dedup-threshold 12 Pre-dedup Hamming distance for cost control
--max-height 720px Maximum download resolution
--keep-temp Keep intermediate frames and crops

Sources