browser-use/video-use

What it solves

Video editing is typically a manual, time-consuming process involving complex software menus and timelines. video-use allows users to edit videos using natural language via AI agents (like Claude Code), removing the need for manual cutting, filler word removal, and tedious formatting tasks.

How it works

Instead of processing every video frame—which would be computationally expensive and noisy—the system uses a two-layer approach to give the LLM a "reading" view of the video:

1. Audio Transcript Layer: It uses ElevenLabs Scribe to generate word-level timestamps, speaker identification, and audio events, condensing the footage into a small text file that the LLM can easily process.
2. Visual Composite Layer: On-demand, the system generates a "timeline view" (a PNG containing a filmstrip, waveform, and word labels) for specific time ranges to help the LLM make precise decisions at cut points or resolve ambiguities.

Once the LLM proposes an editing strategy, it generates an Edit Decision List (EDL) and renders the video using ffmpeg. A self-evaluation loop then checks the rendered output for visual jumps or audio pops before presenting the final result to the user.

Who it’s for

Content creators, educators, and professionals producing talking-head videos, montages, tutorials, travel vlogs, or interviews who want to automate the tedious parts of the editing process.

Highlights

• Automated Cleanup: Automatically removes filler words (e.g., "umm", "uh") and dead space between takes.
• AI-Driven Enhancements: Auto color grading and 30ms audio fades to prevent pops at cuts.
• Dynamic Overlays: Generates animation overlays using tools like HyperFrames, Remotion, Manim, or PIL via parallel sub-agents.
• Customizable Subtitles: Burns in styled subtitles with customizable formatting.
• Self-Evaluating Pipeline: Automatically checks rendered cuts for errors and re-renders if issues are found.
• Session Persistence: Saves project state in a markdown file so editing sessions can be resumed later.