Ollama 0.30 Release Adds GGUF Model Support and Up to 20% Faster NVIDIA Performance

TL;DR

Ollama 0.30 adds native GGUF model support using llama.cpp and boosts NVIDIA GPU throughput by up to 20%, while enabling Vulkan by default to broaden GPU acceleration to AMD and Intel hardware.

Faster NVIDIA Throughput

Ollama 0.30 improves performance on NVIDIA GPUs by as much as 20% thanks to optimizations contributed by the NVIDIA and llama.cpp teams. The benchmark shown uses the Gemma 4 26B model quantized to Q4_K_M on an RTX 5090, demonstrating the higher token‑per‑second rate.

Tested with the Gemma 4 26B model running on an NVIDIA RTX 5090 using the Q4_K_M quantization.

Default Vulkan for Wider GPU Compatibility

Vulkan is now enabled by default in Ollama, extending GPU acceleration beyond Apple silicon to AMD and Intel devices. Users no longer need to install vendor‑specific libraries; models run on any Vulkan‑compatible GPU out of the box.

Expanded Model Compatibility via GGUF

Ollama 0.30 integrates the GGUF ecosystem through llama.cpp, allowing a broader set of models to run without custom conversion steps. Supported families include:

LFM (e.g., LiquidAI/LFM2‑8B‑A1B‑GGUF)
Prism (e.g., prism‑ml/Bonsai‑8B‑gguf)
Fine‑tuned models from Unsloth

How to Run a GGUF Model from Hugging Face

Download the GGUF file or directory.
Create a Modelfile that points to the GGUF path:
```
FROM ./my-model.Q4_K_M.gguf
```

Build and run the model with Ollama:

ollama create -f Modelfile my-model
ollama run my-model

Tool‑Calling Capability Preserved

If a GGUF model includes the tools capability, Ollama retains tool‑calling support, enabling seamless use with coding agents and personal assistants.

Example Launch Commands

Claude Code: ollama launch claude --model my-model
Hermes Agent: ollama launch hermes --model my-model
OpenClaw: ollama launch openclaw --model my-model

To confirm tool support, run:

ollama show my-model

The output will list a tools capability when present.

Acknowledgements

The release credits Georgi Gerganov and the llama.cpp maintainers, as well as hardware partners NVIDIA, AMD, Qualcomm, and Intel, for their contributions to the GGML ecosystem optimizations.

Community Feedback

For questions, feedback, or troubleshooting, users are invited to join the Ollama Discord or email hello@ollama.com.