MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark

What it solves

This project provides a set of optimized, ready-to-run Docker scripts to serve the Qwen3.8-27B model on NVIDIA DGX Spark (GB10) hardware. It eliminates the guesswork of manual tuning by providing pre-measured configurations for different speculative decoding engines, ensuring maximum throughput and stability on this specific architecture.

How it works

The repository uses SGLang as the inference engine and wraps it in Docker containers. It offers three distinct serving modes based on the speculative decoding method used to accelerate token generation:

• EAGLE/MTP: Uses an in-checkpoint head for speculative decoding.
• DSpark: Employs a separate draft model to accelerate code and general chat.
• DFlash2: Uses a block-diffusion draft model (requiring a custom-built image via provided patches) to optimize for both code and long-form writing.

To maximize performance on the GB10's ARM architecture, the scripts pin the process to the high-performance Cortex-X5 cores and utilize FP8 KV caching to save memory. It also supports context window extension up to 1 million tokens via YaRN scaling.

Who it’s for

Developers and researchers using NVIDIA DGX Spark (GB10) hardware who want to deploy Qwen3.8-27B with optimal performance without manually tuning SGLang flags and memory fractions.

Highlights

• Three Speculative Engines: Support for MTP, DSpark, and DFlash2, each tuned for different workloads (e.g., DSpark for code, MTP for long essays).
• Hardware-Specific Tuning: Pinned to Cortex-X5 cores for a 2-7% decode speedup and optimized GDN state pools.
• Flexible Context: Native 262K context support with optional extension up to 1M tokens using YaRN.
• Automated Deployment: Idempotent start/stop scripts and automatic Docker image building for DFlash2 support.