Qwen3.8-Omni-Flash Release Notes

Qwen3.8-Omni-Flash is a native omnimodal model designed to transition AI agents from simple content understanding to autonomous task planning and execution in audio-visual environments. It introduces a 1M-token context window and significant cost reductions, with audio input prices decreasing by over 98% and audio-visual input prices decreasing by over 93% compared to previous versions.

Agentic Audio-Visual Understanding

Qwen3.8-Omni-Flash shifts from static processing to an agentic workflow for long-form content. Instead of processing every frame of a multi-hour video, the model independently decides which segments to watch and listen to through multiple rounds of coarse-to-fine evidence gathering.

On the OmniVideoBench benchmark, this agentic approach improved accuracy from 63.4 to 67.8 while reducing token consumption by approximately 45.7% (from 145,736 to 79,117 tokens per query).

Long-Form Capabilities

  • Controllable Captioning: Users can specify the subject, time range, and level of detail for video descriptions, allowing for structured analysis of camera shots, lighting, and sound.
  • Meeting Analysis: The model supports up to one hour of audio-visual input, performing speaker segmentation, identity alignment, and transcription end-to-end to generate meeting minutes and action items.
  • Deep Research: The model can combine video content with web searches across multimodal sources to produce illustrated research reports.

Audio-Visual Production and Editing

The model enables end-to-end workflows for professional content creation by planning and calling tools to deliver finished media:

  • Music2MV: Analyzes song rhythm, mood, and vocals to design characters and scenes for music videos, outputting line-level lyrics with timestamps.
  • Short Drama Translation: Automates speaker-aware dialogue recognition, voice cloning, dubbing, and audio remixing for international localization.
  • Film Commentary: Extracts key plots from full-length films to plan and produce commentary videos, including automated adjustment of speech rate and volume for natural flow.

Model Optimization and Knowledge Extraction

Qwen3.8-Omni-Flash can be used to drive the development of smaller, specialized models. In one experiment, the model was tasked with improving Qwen2.5-Omni-3B's Sichuan dialect speech recognition. Over 12 hours, the agent independently diagnosed problems and constructed 3,413 training examples, reducing the character error rate from 25.79% to 15.30% (a 40.7% relative reduction).

Additionally, the model supports transforming unstructured video into structured assets:

  • Video2Note: Converts video tutorials into PDF notes with key screenshots and step-by-step instructions.
  • Omni Skill Creator: Extracts standard operating procedures (SOPs) from demonstration videos to create reusable Skill.md files for other agents.

Real-Time Interaction with Qwen3.8-Omni-Flash-Realtime

For low-latency scenarios, the Qwen3.8-Omni-Flash-Realtime model perceives and responds to live audio-visual streams. Key capabilities include:

  • Real-Time Speaking Practice: Jointly models pronunciation and semantics to correct accents and tonal deviations in real time.
  • Spatial Audio Perception: Combines spatial sound with visual data to locate sound sources and navigate toward them, enabling robot navigation based on audio cues.
  • Dynamic Knowledge Injection: Supports the injection of identity settings and business rules via "Skills" for customized customer service interactions.

Performance Benchmarks

Qwen3.8-Omni-Flash demonstrates competitive performance against Gemini 3.8 Flash and other models across several dimensions:

Benchmark Qwen3.8-Omni-Flash Gemini 3.8 Flash
WildClawBench-MM (Tool Use) 71.0 58.9
UniClawBench (Tool Use) 69.6 69.0
DailyOmni (AV Understanding) 85.1 84.0
OmniVideoBench (AV Reasoning) 63.4 65.2
LongAudioSpan (Accuracy) 82.7 79.3

In text-based coding and agent tasks, it scores 63.3 on SWE-bench Pro and 75.3 on CoWorkBench, maintaining text performance comparable to text-only models of the same size.

Ecosystem and Tooling

  • Qwen-MM-Plugins: A multimodal plugin suite supporting image, audio, video, and document understanding. It integrates with harnesses like Claude Code, Qwen Code, and OpenClaw.
  • Qwen-Live Harness: An open-source runtime for continuous, real-time omnimodal interaction, supporting task delegation and long-term memory.

Community Insights

Users on Hacker News highlighted the potential for massive cost reductions compared to competitors. One user noted:

"If the performances are comparable... in/out ($) Gemini : 1.5 / 9.0 | Qwen 3.8: 0.15 / 0.47 That is a massive cost reduction."

However, some users expressed skepticism regarding the presentation of the blog post and reported issues with the accessibility of the harness repository links.

Sources

Related