QVQ-72B-Preview Release

Qwen has introduced QVQ-72B-Preview, an open-weight model designed for complex multimodal reasoning. Built upon Qwen2-VL-72B, QVQ-72B-Preview is engineered to enhance AI's capacity for visual understanding and analytical problem-solving, particularly in domains requiring sophisticated step-by-step reasoning.

Performance Benchmarks

QVQ-72B-Preview demonstrates significant improvements in visual reasoning over its predecessor, Qwen2-VL-72B-Instruct. The model was evaluated across four primary datasets:

  • MMMU: A university-level multidisciplinary multimodal evaluation dataset. QVQ-72B-Preview achieved a score of 70.3, significantly outpacing Qwen2-VL-72B-Instruct.
  • MathVista: A mathematics-focused visual reasoning test set covering puzzle graphics, function graphs, and academic figures.
  • MathVision: A multimodal mathematical reasoning test set derived from real mathematics competitions.
  • OlympiadBench: A bilingual multimodal science benchmark containing 8,476 problems from Olympic mathematics and physics competitions, including the Chinese college entrance examination.

In mathematics and science benchmarks, QVQ-72B-Preview effectively closes the performance gap with the state-of-the-art o1 model.

Technical Capabilities and Reasoning Process

QVQ-72B-Preview employs a methodical, step-by-step reasoning approach to solve complex visual tasks. Demo cases illustrate its ability to handle diverse technical challenges:

  • Mathematics: The model can apply the product rule for derivatives to solve table-based calculus problems and use circle formulas to evaluate integrals of graphed functions.
  • Physical Volume Calculation: The model can decompose complex L-shaped objects into rectangular prisms and account for overlapping volumes to calculate total volume.
  • Chemistry: The model can analyze chemical flowcharts (e.g., copper extraction from chalcopyrite) to identify specific chemical substances based on reaction sequences.
  • Biology: The model can analyze genetic diagrams to determine the cause of chromosomal abnormalities, such as Klinefelter syndrome, by evaluating allele inheritance.
  • Geometry: The model can use coordinate geometry and linear equations to calculate the probability of a random point falling within a specific triangle inside a rectangle.

Model Limitations

As an experimental research model, QVQ-72B-Preview has several known limitations:

  • Language and Logic: The model may exhibit unexpected language mixing, code-switching, or get stuck in recursive reasoning patterns (circular logic) that produce verbose responses without reaching a conclusion.
  • Reliability: During multi-step visual reasoning, the model may lose focus on the image content, which can lead to hallucinations.
  • Functional Scope: It is not intended to fully replace the capabilities of Qwen2-VL-72B-Instruct.
  • Safety: The model requires enhanced safety measures for secure deployment.

Future Development

Qwen aims to evolve QVQ into an "omni and smart" model. Future development will focus on integrating additional modalities into a unified model to enhance its capabilities in scientific exploration and the resolution of complex challenges.

Sources