Entrpi/ds4-on-spark

Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark, and full continuous batch support

What it solves

This project provides a streamlined way to deploy and serve the DeepSeek-V4-Flash model on NVIDIA DGX Spark hardware. It addresses the challenge of maximizing throughput and context window utilization on unified-memory devices, offering a high-performance alternative to the upstream engine by adding batched serving, continuous batching, and advanced memory management.

How it works

The project uses a fork of the ds4 engine (DwarfStar 4) optimized for Blackwell CUDA (sm_121). It implements a "memory governor" that avoids pre-reserving memory; instead, it demand-maps context and measures each request's actual memory needs against live system free memory.

Key technical components include:

  • Speculative Decoding: Uses DSpark for lossless speculative decoding to increase speed.
  • KV Cache Management: Implements prefix caching with disk-persisted KV banks that survive restarts.
  • API Compatibility: Supports OpenAI (Chat/Completions/Responses) and Anthropic Messages APIs, allowing it to work with agents like Claude Code and OpenAI Codex.
  • Memory Governor: Dynamically manages memory to allow up to 3 million tokens of active context on a single Spark device without crashing, providing typed refusals when memory is exhausted.

Who it’s for

Developers and researchers using NVIDIA DGX Spark (GB10/SM121) or high-end Blackwell GPUs (RTX PRO 6000 / 5090-class) who need to serve DeepSeek-V4-Flash with massive context windows (up to 1M tokens) and high throughput for agentic workflows.

Highlights

  • High Performance: 2.4-3.3x prefill throughput and 1.33-1.47x decode speed compared to the upstream engine.
  • Massive Context: Supports the model's full 1M token window and can maintain up to 3 million tokens of active resident context.
  • Agent Optimization: Includes protocol reminders for tool-use at depth and options to drop reasoning echoes to reduce prompt length by 16-28%.
  • One-Command Setup: A single installation script handles host verification, building the engine, downloading GGUF models, and launching the server.

Related

  • Project
    MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark

    A vLLM-based serving recipe for DeepSeek-V4-Flash on 2× DGX Spark, using DSpark speculative decoding, NVFP4 quantization, and a 1M-token context window.

  • Project
    antirez/ds4

    A native inference engine optimized for DeepSeek V4 and GLM 5.2, featuring SSD streaming and speculative decoding to run large models on consumer hardware.

  • Dispatch
    DeepSeek V4 Flash runs on a single AMD MI300X – performance, fixes, and deployment guide

    DeepSeek V4 Flash can be served on a single AMD MI300X GPU with 168 tok/s decode throughput and 8 K tok/s prefill without quantization, thanks to a set of ROCm patches, AITER tuning, and a hybrid KV cache.

  • Project
    MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark

    Optimized Docker scripts for serving Qwen3.8-27B on NVIDIA DGX Spark using SGLang, featuring pre-tuned configurations for MTP, DSpark, and DFlash2 speculative decoding.

  • Dispatch
    DeepSeek-V4 Preview Release

    DeepSeek has released the open-source DeepSeek-V4 preview, introducing two model variants—Pro and Flash—featuring a 1M token context window and state-of-the-art agentic coding capabilities.