Gemini Agentic Video Understanding Release

Google DeepMind has introduced agentic video understanding for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. This capability replaces static frame-rate processing with a goal-directed agentic loop that dynamically searches and inspects video segments, reducing token consumption by up to 88% and costs by up to 66% while increasing accuracy by up to 7%.

Dynamic Processing vs. Static Analysis

Agentic video understanding shifts video analysis from a fixed frames-per-second (FPS) ingestion model to a dynamic, tool-based approach. In standard static processing, models ingest video at a fixed rate (defaulting to 1 FPS). In contrast, agentic video understanding allows Gemini to actively determine what to watch, the speed of playback, and the modality to use—including visual frames, audio, and transcripts—fetching only the necessary signals to answer a query.

This approach reduces development overhead by automating the process of loading relevant video segments via an internal tool, a task that previously required manual implementation by developers.

Performance Benchmarks and Efficiency

Across standard video analysis benchmarks, the agentic approach provides significant efficiency gains, particularly for long-form content such as multi-hour recordings, 90-minute lectures, and 10-minute how-to guides.

Key performance metrics include:

  • Token Consumption: Reduced by up to 88%.
  • Analysis Costs: Reduced by up to 66%.
  • Accuracy: Improved by up to 7%.

Gemini 3.7 Flash is identified as the model providing the best overall quality and the most efficient combination of quality and cost, placing it at the accuracy-to-cost pareto frontier for video analysis.

Core Capabilities and Use Cases

Agentic video understanding enables high-precision tasks that are often impossible or cost-prohibitive under static 1 FPS processing:

  • Sub-second Moment Retrieval: The ability to pinpoint split-second state changes and tight cut boundaries for automated video editing.
  • Long-form Needle-in-a-Haystack Search: Executing complex queries across multi-hour videos without consuming millions of tokens.
  • Anomaly Detection: The ability to resample specific time windows at higher FPS to inspect subtle visual artifacts and rapid motion.
  • Counting Actions and Objects: Precise tracking of repeated physical movements and distinct objects over time.

Availability and Implementation

Agentic video understanding is available via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. It is integrated into Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite and follows standard Gemini API token pricing with no additional feature fee.

Developers can enable the feature by setting the processing parameter to "agentic" in the API configuration.

Beyond the API, this technology is being integrated into the Gemini app across Flash and Flash-Lite models and will power the "Ask YouTube" feature on the video watch page in the coming months.

Sources