DeepSeek-V3-0324 Release Notes

DeepSeek has released DeepSeek-V3-0324, an updated version of the base model underlying the R1 reasoning model. This release focuses on enhancing instruction following, mathematical reasoning, and coding capabilities while transitioning the model to an MIT license.

Performance Benchmarks and Capabilities

DeepSeek-V3-0324 demonstrates strong performance across key benchmarks, frequently performing on par with GPT-4.5 and generally exceeding Claude-Sonnet-3.7.

Benchmark Improvements

The model shows significant gains over previous versions in the following areas:

  • MMLU-Pro: Increased from 75.9 to 81.2 (+5.3), indicating improved overall understanding.
  • GPQA: Increased from 59.1 to 68.4 (+9.3).
  • AIME: Increased from 39.6 to 59.4 (+19.8), serving as a proxy for math capabilities.
  • LiveCodeBench: Increased from 39.2 to 49.2 (+10.0), indicating improved coding abilities.

Targeted Feature Enhancements

DeepSeek identified specific improvements in the following domains:

  • Front-End Web Development: Enhanced code executability and the aesthetic quality of game front-ends and web pages.
  • Chinese Writing Proficiency: Improved style and content quality for medium-to-long-form writing, aligned with the R1 writing style, and optimized translation and letter writing.
  • Chinese Search Capabilities: More detailed outputs for report analysis requests.
  • Function Calling: Increased accuracy and the resolution of issues present in previous V3 versions.

Technical Implementation

While a full technical report has not been released, the model maintains the same architecture as the original DeepSeek-V3. It is hypothesized that these improvements were achieved through:

  • Continual Pretraining: Utilizing newer, higher-quality, and better-curated data to improve general capabilities and factuality regarding recent events.
  • Improved Post-Training: Refining the post-training data mix or algorithms to enhance instruction following and style.

Deployment and Inference Options

DeepSeek-V3-0324 can be deployed through several methods depending on the available compute resources:

Managed Inference

The model is available via Hugging Face Inference Providers, including Fireworks, Hyperbolic, and Novita.

Self-Hosted Inference

  • Text Generation Inference (TGI): Supported in the latest release (v3.2.1) and can be run via Docker on H100 nodes.
  • SGLang: Supports the model out of the box with Multi Latent Attention and Data Parallelism optimizations.

Quantization for Lower VRAM

To reduce the high VRAM requirements of the full model, Unsloth AI has provided Dynamic Quantizations. These allow the model to run with half the compute of a single H100 node via llama.cpp with minimal degradation in benchmark performance.

Security and Safety Considerations

Model Distribution Safety

Downloading and running the model is considered safe due to several Hugging Face Hub safeguards:

  • Safetensors: The use of the safetensors format prevents hidden code execution, a risk associated with the older PyTorch pickle format.
  • Modeling Code Security: Code is fully visible on the hub, requires the user to set trust_remote_code=True for execution, and is subject to a security scanner. Users can further secure their environment by pinning the model version using the revision setting.

Output and Usage Risks

Users should apply standard safety measures when utilizing model outputs:

  • Alignment: Users should be aware that all LLMs possess biases based on the model provider's alignment values.
  • Code Generation: Proposed code should undergo thorough review and vulnerability scanning, as models may generate code containing known vulnerabilities found in their training data.
  • Agents: When using the model in autonomous agents, it is recommended to use sandboxes to prevent unauthorized system access, avoid sharing private credentials, and maintain a human-in-the-loop for high-stakes processes.

Sources