Gemma 4 12B release notes / what's new

Google has released Gemma 4 12B, a mid-sized multimodal model designed for local execution on consumer hardware. The model bridges the gap between the edge-optimized E4B and the larger 26B Mixture of Experts (MoE) model, providing high-performance reasoning and agentic capabilities with a reduced memory footprint.

Unified Encoder-Free Architecture

Gemma 4 12B eliminates traditional multimodal encoders to reduce latency and memory overhead. Instead of using separate models to translate images and audio into representations for the LLM, the model integrates these inputs directly into the LLM backbone.

  • Vision Processing: The standard vision encoder is replaced by a lightweight embedding module consisting of a single matrix multiplication, positional embedding, and normalizations.
  • Audio Processing: The audio encoder is removed entirely. Raw audio signals are projected directly into the same dimensional space as text tokens.

Local Deployment and Hardware Requirements

Gemma 4 12B is optimized for laptops with 16GB of VRAM or unified memory. This allows developers to run state-of-the-art multimodal agents locally without relying on cloud infrastructure.

To support low-latency inference, the model includes Multi-Token Prediction (MTP) drafters. It is released under an Apache 2.0 license, ensuring broad accessibility across the developer ecosystem.

Integration and Tooling

Developers can integrate the model using several industry-standard tools:

  • Inference Engines: llama.cpp, vLLM, SGLang, and MLX.
  • Local LLM Apps: LM Studio and Ollama.
  • Fine-tuning: Unsloth.
  • Deployment: Google Cloud (Cloud Run, GKE, and Gemini Enterprise Agent Platform Model Garden).

Performance and Community Feedback

Gemma 4 12B delivers reasoning performance nearing the 26B MoE model on standard benchmarks while using less than half the memory. However, early community testing reveals a mixed reception regarding its real-world multimodal accuracy.

Technical Insights from the Community

Users have reported varying results in vision and audio tasks. Some users found the vision capabilities to be inferior to much smaller models like Qwen 3.5 0.8B in specific tests, while others noted failures in identifying specific objects (e.g., the Taj Mahal) that larger Gemma models handled correctly.

Regarding coding, some users reported that the model is a capable local coding assistant, comparing its output to GPT-4.1 in specific benchmarks, though some noted occasional syntax errors such as extra closing brackets.

Memory and Quantization Considerations

While Google states the model runs on 16GB of RAM, community members have pointed out that this likely requires quantization. The original weights are released in 16-bit float (BF16), which would exceed 16GB of memory. Users have successfully run 4-bit (Q4) and 8-bit (Q8) quantizations via llama.cpp, with some reporting generation speeds around 25-72 tokens per second depending on the hardware (e.g., RTX 4080 Super).

"I ran the Q4 quant... The result is decent, but it had a few bizarre/trivial syntax errors I had to fix manually... it roughly compares with GPT-4.1... on the output."

Summary of Model Capabilities

Feature Gemma 4 12B
Architecture Unified, Encoder-Free
Primary Hardware Target Laptops (16GB RAM/VRAM)
License Apache 2.0
Key Modalities Text, Vision, Audio (Native)
Optimization Multi-Token Prediction (MTP)

Sources