MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark

This project is a deployment recipe (not a new model) for running DeepSeek-V4-Flash-0731 on a 2-node NVIDIA DGX Spark cluster. It wires together vLLM, DSpark speculative decoding, NVFP4 quantization, and RoCE networking to serve the model with a 1M-token context window.

Key features:

• 2-node tensor parallel (TP=2) with NCCL/RoCE, plus a worker/head architecture.
• DSpark speculative decoding (5 draft tokens) for faster generation.
• NVFP4 KV cache (nvfp4_ds_mla) to fit a huge context in memory.
• 1M context ceiling with 6 concurrent sequences (safe because the KV pool is ~2.49M tokens).
• Hotfixes for known vLLM issues: stop strings firing inside reasoning, tool-call truncation, prefill starvation, and CPU spin.
• Optional vision sidecar (Qwen3-VL-4B) and a Stage-C profile for higher throughput.

Performance (default Anemll stack):

• Single chat: ~62–83 tok/s after first token.
• Six short concurrent chats: ~160–190 tok/s aggregate (~30–37 per stream).
• Stage-C profile (200K context, 16 slots): 315 tok/s static.

Usage: Copy .env.dspark.example → .env.dspark, set cluster IPs, pull the Docker image, download weights, then start the worker before the head node. The README includes detailed env-var tables, troubleshooting, and benchmark results.