Ollama New Model Scheduling Update

Ollama has released a new model scheduling system that replaces memory estimation with exact memory measurement. This update improves system stability by reducing out-of-memory crashes and increases performance by maximizing GPU utilization and optimizing multi-GPU scheduling.

Exact Memory Measurement and System Stability

Ollama's new engine now calculates the exact amount of memory required to run a model rather than relying on estimations. This shift to exact memory management eliminates over-allocations, which significantly reduces the frequency of crashes caused by out-of-memory (OOM) issues.

GPU Utilization and Performance Gains

The new scheduling system maximizes GPU utilization by allocating more memory to the GPU, which directly increases both token generation and prompt processing speeds. This optimization extends to multi-GPU setups, where Ollama now schedules models more efficiently across multiple cards, improving performance for both multi-GPU and mismatched GPU configurations.

Performance Benchmarks

In tests using a single NVIDIA GeForce RTX 4090, the gemma3:12b model with a 128k context length showed a significant increase in token generation speed, rising from 52.02 tokens/s to 85.54 tokens/s, while VRAM usage increased from 19.9GiB to 21.4GiB and all 49 layers were loaded onto the GPU.

For image input tasks using two NVIDIA GeForce RTX 4090 GPUs with the mistral-small3.2 model and a 32k context length, prompt evaluation speed increased from 127.84 tokens/s to 1380.24 tokens/s, and token generation speed rose from 43.15 tokens/s to 55.61 tokens/s. In this configuration, all 41 layers and the vision model were loaded onto the GPU.

System Monitoring and Reporting

Memory reporting is now synchronized across tools. Measurements displayed in ollama ps will now match those reported by nvidia-smi, providing users with a consistent and accurate way to track memory utilization on their systems.

Model Support and Availability

This new memory management feature is enabled by default for all models implemented in Ollama's new engine. Supported models include:

• GPT-OSS: gpt-oss
• Llama: llama4, llama3.2-vision (with llama3.2, llama3.1, and llama3 coming soon)
• Gemma: gemma3, embeddinggemma, gemma3n
• Qwen: qwen3, qwen2.5vl (with qwen3-coder coming soon)
• Mistral: mistral-small3.2
• Embedding Models: all-minilm and other embedding models