Gemma 3n Release Notes / What's New

Google DeepMind has released Gemma 3n, a mobile-first multimodal model family designed to bring frontier-level capabilities to edge devices. The release introduces a nested transformer architecture that allows models to run with a memory footprint significantly smaller than their raw parameter count, enabling high-performance multimodal AI on hardware with as little as 2GB of RAM.

Multimodal Capabilities and Performance

Gemma 3n natively supports image, audio, video, and text inputs with text outputs. It is available in two primary sizes based on effective parameters: E2B and E4B. While the raw parameter counts are 5B and 8B respectively, architectural optimizations allow them to operate with memory footprints of 2GB (E2B) and 3GB (E4B).

Key performance milestones include:

  • LMArena Score: The E4B version is the first model under 10 billion parameters to achieve an LMArena score over 1300.
  • Multilinguality: The models support text in 140 languages and multimodal understanding in 35 languages.
  • General Improvements: Enhanced quality in reasoning, coding, and mathematics.

MatFormer: Elastic Inference Architecture

Gemma 3n is built on the MatFormer (Matryoshka Transformer) architecture, which utilizes nested transformers to enable elastic inference. In this design, a larger model contains smaller, fully functional versions of itself.

Developers can utilize this architecture in two ways:

  1. Pre-extracted Models: Users can download the standalone E2B sub-model for up to 2x faster inference compared to the E4B model.
  2. Mix-n-Match Customization: Using the MatFormer Lab tool, developers can create custom-sized models between E2B and E4B by adjusting the feed-forward network hidden dimension per layer (ranging from 8192 to 16384) and selectively skipping layers.

While not yet implemented in the current launch, this architecture enables future "elastic execution," allowing a single deployed model to dynamically switch between E4B and E2B inference paths based on device load and task requirements.

Memory and Processing Optimizations

Per-Layer Embeddings (PLE)

Per-Layer Embeddings (PLE) improve memory efficiency by allowing a significant portion of the model's parameters—specifically the embeddings associated with each layer—to be computed on the CPU. This ensures that only the core transformer weights (approximately 2B for E2B and 4B for E4B) occupy the constrained accelerator memory (VRAM).

KV Cache Sharing

To accelerate the processing of long sequences from audio and video streams, Gemma 3n implements KV Cache Sharing. By sharing the keys and values of the middle layer from local and global attention with all top layers, the model achieves a 2x improvement in prefill performance compared to Gemma 3 4B, reducing the time-to-first-token for streaming applications.

Specialized Multimodal Encoders

Audio Understanding

Gemma 3n integrates an audio encoder based on the Universal Speech Model (USM), generating one token every 160ms of audio. This enables on-device Automatic Speech Recognition (ASR) and Automatic Speech Translation (AST), with particularly strong results in translation between English and Spanish, French, Italian, and Portuguese.

MobileNet-V5 Vision Encoder

The model utilizes the MobileNet-V5-300M vision encoder, which is optimized for edge devices. Key features include:

  • Throughput: Capable of processing up to 60 frames per second on a Google Pixel.
  • Flexibility: Supports input resolutions of 256x256, 512x512, and 768x768 pixels.
  • Efficiency: Compared to the baseline SoViT in Gemma 3, MobileNet-V5-300M delivers a 13x speedup with quantization (6.5x without), uses 46% fewer parameters, and has a 4x smaller memory footprint.

Ecosystem and Deployment

Gemma 3n is supported by a wide array of open-source tools and frameworks, including Hugging Face Transformers, llama.cpp, Ollama, MLX, Google AI Edge, and vLLM. Deployment options range from Google AI Studio and Vertex AI to the NVIDIA API Catalog and Cloud Run.

Sources