Unsloth Dynamic 3.0 GGUF Release

Unsloth Dynamic 3.0 delivers higher accuracy at equivalent sizes

Unsloth Dynamic v3.0 is a significant upgrade over Dynamic v2.0, specifically designed to preserve more model quality while maintaining the same disk footprint. For the Qwen3.8-27B model, Dynamic 3.0 quants deliver more than 10% better top-1% accuracy compared to every other provider at the same size. These GGUFs are compatible with most inference engines, including llama.cpp and Unsloth Desktop.

Key technical improvements in Dynamic 3.0

Dynamic 3.0 achieves its performance gains through several methodological shifts in post-training quantization (PTQ):

High-Quality Calibration Datasets: Unsloth now utilizes a refined imatrix calibration dataset sourced from diverse origins, specifically optimized for multilingual performance, chat, and agentic coding.
Enhanced Layer Selection: The process for selecting which layers to quantize and how to do so has been improved to maximize quality preservation.
Pure PTQ Approach: Unsloth explicitly does not use Quantization Aware Training (QAT) or Quantization Aware Distillation (QAD), nor do they train on the imatrix calibration dataset. This approach reduces the risk of overfitting compared to QAD/QAT methods.

Model-specific optimizations and size reductions

To optimize for disk space and performance, Unsloth has implemented specific changes to the Qwen3.8-27B quants:

MTP Module Removal: For smaller quants under UD-Q2_K_XL (8.37GB and lower), the MTP module was removed to save approximately 500MB of disk space. Users requiring MTP can still use the separate Q4_0 MTP module.
Extreme Compression: The UD-IQ1_S quant is 6.2GB (without MTP), retaining approximately 72% top-1% accuracy while being 89% smaller than the full model.
Improved Low-Bit Performance: The UD-Q2_K_XL quant (9.83GB) is approximately 8% more accurate on top-1% than the next best alternative and has demonstrated the ability to generate working HTML programs that previously failed.

Benchmarking Methodology: Divergence-300 and KL Divergence

Unsloth argues that standard top-1% accuracy (argmax on one prediction) is insufficient for gauging actual inference performance. To address this, they introduced two primary metrics:

Divergence-300 @32

This metric uses a held-out dataset of 300 examples from sources like Terminal-Bench 2.1, DeepSWE, Harbor, and MathArena 2025-26. The process involves greedy argmax decoding for 32 tokens and comparing the trajectories of the quantized model against the BF16 (full precision) version. This determines if the quantized model's output remains similar to the original over multiple tokens, providing a better gauge of overfitting than single-token accuracy.

KL Divergence (KLD)

Unsloth identifies KL Divergence as a gold standard for reporting quantization errors because it measures "flips"—instances where an answer changes from incorrect to correct or vice versa. They argue that perplexity is an incorrect metric because output token values can cancel each other out. To prevent overfitting, Unsloth benchmarks KLD on unseen datasets (such as Wikitext and Code) rather than the calibration dataset.

Comparison with Google's Gemma 3 QAT

Unsloth benchmarked their Dynamic quants against Google's official Quantization Aware Training (QAT) versions of Gemma 3. For the Gemma 3 (27B) model, Unsloth found that their dynamic 4-bit version was 2GB smaller than the Google QAT version while providing +1% extra accuracy on 5-shot MMLU.

Unsloth also introduced an Efficiency Metric to evaluate the utility of a model relative to its size:

$$\text{Efficiency} = \frac{\text{MMLU 5-shot score} - 25}{\text{Disk Space GB}}$$

This formula subtracts 25 from the score to account for the random-chance baseline of a 4-choice MMLU test.

Llama 4 Bug Fixes and Integration

Unsloth has contributed several critical fixes to the open-source ecosystem to improve Llama 4 performance:

RoPE Scaling: Resolved issues in llama.cpp to support changes in the Llama 4 Scout RoPE Scaling configuration.
QK Norm Epsilon: Fixed the QK Norm epsilon for Scout and Maverick in both llama.cpp and transformers, ensuring it uses 1e-05 instead of 1e-06.
QK Norm Sharing: Collaborated with the Llama 4 team and vLLM to fix an issue where QK Norm was shared across all heads, which increased MMLU Pro accuracy from 68.58% to 71.53%.

Community Insights and Feedback

Community discussion around the release highlighted several practical concerns and requests:

Version Control: Users noted that GGUF files often share the same name across different versions (e.g., Dynamic 2.0 vs 3.0), making it difficult to manage local storage without explicit version numbers in filenames.
MTP Performance: Some users reported that the Multi-Token Prediction (MTP) module can actually slow down inference on certain hardware, such as the M4 Max MacBook Pro.
Format Requests: There is significant demand for MLX versions of the Qwen3.8-27B quants for better integration with Apple Silicon.

"The one downloaded 3 or 4 days ago is a different thing and is NOT the 'Dynamic 3.0' GGUF which I am now downloading..." — @walrus01