Ollama MLX Engine Update for Apple Silicon

Ollama has updated its MLX engine to deliver higher performance on Apple Silicon by leveraging Apple’s unified memory and the Metal-backed MLX framework. These updates result in higher quality responses, increased output speeds, and reduced memory usage.

Higher Quality Inference with NVFP4 Support

Ollama’s MLX engine now supports NVIDIA’s model-optimized NVFP4 format. This 4-bit quantization format reduces quality loss compared to other 4-bit formats by more closely tracking the local dynamic range of model weights.

For the Gemma 4 12B model, NVFP4 roughly halves the quality loss (measured by perplexity) relative to unquantized BF16 weights when compared to the q4_K_M quantization format. Additionally, this support enables portability between datacenters and desktops, as models optimized for datacenter deployment can now be imported and run via the MLX engine.

Increased Output Performance

The updated MLX engine provides up to a 20% increase in output speed. This performance gain is achieved through two primary technical optimizations:

Kernel Fusion: Several operations are now fused into single Metal kernels using the just-in-time (JIT) compiler features of MLX.
Sampling Efficiency: Ollama has reworked its GPU-backed sampling to operate more efficiently.

Benchmarks indicate that NVFP4 generates tokens approximately 20% faster than q4_K_M on the updated engine, based on average output speeds over 10 runs with an 8,300-token input prompt.

Optimized Agent Workflows via State Snapshots

To address the overhead of prompt processing in agentic workloads—where tool calls and repeated transcripts often force the model to re-process the same context dozens of times—Ollama has introduced a snapshot system. This system saves model state at key points to avoid redundant processing.

Applications of the Snapshot System

Multiple Agents: When an agent hands off to a sub-agent or runs multiple concurrent sessions, each resumes from its own saved state. Common context, such as system prompts and tool definitions, is only processed once.
Thinking Models: For reasoning models that generate tokens which are later dropped from the history, snapshots taken immediately before a response starts allow the next turn to resume without reprocessing the entire conversation.
Branching and Retries: When a user regenerates a response or takes a different follow-up path, the engine resumes from the snapshot where the conversation diverged rather than reprocessing the entire history.

Technical Implementation

Because sliding-window attention and recurrent layers carry state that cannot be rewound, Ollama saves state selectively and incrementally at critical points: where conversations branch, at intervals throughout long prompts, and immediately before each response. This selective approach preserves memory for the model while ensuring responsiveness.

Implementation and Usage

Users can access these improvements by downloading the latest version of Ollama and running MLX-optimized models. For example, to run Gemma 4 12B with the MLX engine:

ollama run gemma4:12b-mlx

For integration into a coding agent, the ollama launch command is used:

ollama launch pi --model gemma4:12b-mlx