Granite 4.0 3B Vision release notes / what's new

Granite 4.0 3B Vision release notes / what's new

TL;DR

IBM has released Granite 4.0 3B Vision, a compact vision-language model (VLM) specifically engineered for enterprise document understanding. It provides high-precision extraction of tables, charts, and semantic key-value pairs (KVPs) from complex documents and forms, operating as a modular LoRA adapter on top of the Granite 4.0 Micro language model.

Key Capabilities for Enterprise Documents

Granite 4.0 3B Vision is purpose-built for reliable information extraction from structured visuals and complex layouts. Its primary capabilities include:

  • Table Extraction: Parsing complex table structures, including multi-row and multi-column formats, from document images.
  • Chart Understanding: Converting charts and figures into executable code, summaries, or structured machine-readable formats.
  • Semantic Key-Value Pair (KVP) Extraction: Identifying and grounding semantically meaningful field pairs across diverse document layouts.
  • General Vision-Language Tasks: Producing detailed natural-language descriptions of images.

Technical Architecture and Training

The model's performance is driven by three primary technical investments: a specialized dataset, a modified feature injection architecture, and a modular deployment strategy.

ChartNet: Code-Guided Data Augmentation

To improve spatial precision and numerical reasoning in charts, IBM developed ChartNet, a million-scale multimodal dataset.

ChartNet utilizes a code-guided synthesis pipeline to generate 1.7 million diverse chart samples across 24 chart types and 6 plotting libraries. Each sample contains five aligned components: plotting code, a rendered image, a data table, a natural language summary, and QA pairs. This approach allows the model to learn the relationship between the visual representation and the underlying structured data.

DeepStack: Multi-Layer Visual Feature Injection

Unlike standard VLMs that inject visual information at a single point, Granite 4.0 3B Vision uses a variant of the DeepStack architecture. This method routes abstract visual features into earlier layers for semantic understanding while feeding high-resolution spatial features into later layers. This dual-stream approach ensures the model preserves the fine-grained detail necessary for accurate table and KVP parsing where layout is critical.

Modular LoRA Design

Granite 4.0 3B Vision is implemented as a LoRA adapter on top of the Granite 4.0 Micro dense language model. This modularity allows a single deployment to handle both multimodal and text-only workloads, automatically falling back to the base language model when vision capabilities are not required.

Performance Benchmarks

Granite 4.0 3B Vision demonstrates competitive performance against significantly larger models across several key benchmarks:

Chart Reasoning

Evaluated via LLM-as-a-judge on the ChartNet benchmark:

  • Chart2Summary: Achieved the highest score (86.4%) among all evaluated models.
  • Chart2CSV: Ranked second with a score of 62.1%, trailing only the Qwen3.5-9B model (63.4%).

Table Extraction

Measured by TEDS (which evaluates both structural and content accuracy) across three benchmarks:

  • PubTables-v2: Led performance on both cropped (92.1) and full-page (79.3) settings.
  • OmniDocBench-tables: Achieved a leading score of 64.0.
  • TableVQA-extract: Achieved a leading score of 88.1.

Semantic KVP Extraction

On the VAREX benchmark, which consists of 1,777 U.S. government forms with varying complexity, the model achieved 85.5% Exact Match (EM) accuracy in a zero-shot setting.

Deployment and Integration

Granite 4.0 3B Vision is released under the Apache 2.0 license and can be integrated into workflows in two primary ways:

Stand-Alone Extraction

The model can run directly on individual images for targeted visual extraction, making it suitable for lightweight tools such as form parsers or chart analyzers.

Integrated Pipeline with Docling

When paired with Docling, the model can be used for end-to-end document understanding of multi-page PDFs. In this pipeline, Docling handles OCR and layout parsing to detect and crop visual elements, which are then passed to Granite 4.0 3B Vision for fine-grained extraction (e.g., converting tables to HTML or charts to CSV).

Sources