NVIDIA DGX Spark Performance with Ollama

Ollama has published performance benchmarks for the NVIDIA DGX Spark, showcasing the hardware's ability to run a wide range of large language models (LLMs) using Ollama v0.12.6. These tests demonstrate the throughput capabilities of the GB10 Grace Blackwell Superchip, particularly its capacity to handle large models like gpt-oss-120b within its 120GB of VRAM.

Performance Benchmarks

Ollama's testing on the NVIDIA DGX Spark shows varying prefill and decode speeds across different model architectures and quantization levels. The benchmarks were conducted using firmware version 580.95.05, with temperature set to 0, caching disabled, and output constrained to 500 tokens.

Throughput Results

Device Model name Model size Quantization Prefill (tokens/sec) Decode (tokens/sec)
NVIDIA DGX Spark gpt-oss 20B MXFP4 3.224k 58.27
NVIDIA DGX Spark gpt-oss 120B MXFP4 1.169k 41.14
NVIDIA DGX Spark gemma3 12B q4_K_M 1.894k 24.25
NVIDIA DGX Spark gemma3 12B q8_0 1.406k 15.46
NVIDIA DGX Spark gemma3 27B q4_K_M 834.1 10.83
NVIDIA DGX Spark gemma3 27B q8_0 585.4 7.210
NVIDIA DGX Spark llama3.1 8B q4_K_M 7.614k 38.02
NVIDIA DGX Spark llama3.1 8B q8_0 6.110k 25.23
NVIDIA DGX Spark llama3.1 70B q4_K_M 1.911k 4.423
NVIDIA DGX Spark deepseek-r1 14B q4_K_M 5.919k 19.99
NVIDIA DGX Spark deepseek-r1 14B q8_0 4.667k 1.433
NVIDIA DGX Spark qwen3 32B q4_K_M 705.0 9.411
NVIDIA DGX Spark qwen3 32B q8_0 487.2 6.240

Technical Testing Parameters

To ensure consistency, each test was performed 10 times using a prompt asking for an in-depth summary of the first 200 lines of the book "A Tale of Two Cities". The test script and its associated readme are available for public customization.

Regarding the gpt-oss models, Ollama uses the official versions provided by OpenAI. While some GGUFs found online labeled as MXFP4 are further quantized to q8_0 in the attention layers, Ollama maintains these layers as BF16 as intended by OpenAI.

Hardware and Software Requirements

Optimal performance on the NVIDIA DGX Spark requires specific firmware and software versions. Ollama recommends using the DGX Dashboard for updates if the firmware is below version 580.95.05.

For users preferring the command line interface (CLI), the update process requires upgrading both the Ubuntu distribution and the firmware using the following sequence:

sudo apt update
sudo apt dist-upgrade
sudo fwupdmgr refresh
sudo fwupdmgr upgrade
sudo reboot

Integration with OpenAI Codex

Ollama works seamlessly with OpenAI's Codex. Users can install the Codex CLI via npm (npm install -g @openai/codex) and execute models using the command codex --oss --model gpt-oss.

Because the NVIDIA DGX Spark is powered by the GB10 Grace Blackwell Superchip, it provides 120GB of VRAM, allowing the larger gpt-oss-120b model to fit entirely within the GPU memory:

codex --oss --model gpt-oss:120b

Sources

Related