MiaAI-Lab/DeepSeek-v4-Flash-One-DGX-Spark

What it solves

This project provides a high-performance, single-node deployment launcher for the DeepSeek V4 Flash 0731 model on NVIDIA DGX Spark hardware. It eliminates the need for multi-node setups (tensor-parallel 2) typically required for the official FP4 build, allowing the model to run on a single device with high throughput and massive context windows.

How it works

The project uses a Docker-based recipe tuned for the NVIDIA DGX Spark (GB10, SM121) with 128 GiB of unified memory. It leverages the sparkinfer kernel stack and EXL3 quantization (3.0 bpw) to fit the model on one node. To optimize performance, it implements DSpark K5 speculative decoding with a K64 draft model and uses nvfp4_ds_mla compressed KV cache. It also includes CUDA-graph capture to ensure concurrent decoding stays on optimized graphs rather than falling back to eager execution.

Who it’s for

Developers and researchers with access to NVIDIA DGX Spark hardware who want to deploy DeepSeek V4 Flash with maximum efficiency, long-context capabilities (up to 384k tokens), and OpenAI-compatible API access.

Highlights

• Single-Node Deployment: Runs on one DGX Spark without requiring a second node.
• Massive Context Window: Supports up to 384,000 tokens with verified exact needle-in-a-haystack recall.
• Speculative Decoding: Uses DSpark K5 speculative decoding for faster token generation.
• Optimized Memory: Employs compressed KV cache and tuned GPU memory utilization (0.94) to maximize headroom.
• Runtime Abliteration: Includes an optional, non-destructive runtime projection to remove refusal directions without modifying model weights.
• Local Weight Management: Fully local weight downloading and coalescing, with an optional LAN sharing mode via SSHFS.