MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks

GLM-5.3 Flash EXL3 for 2-4x DGX Sparks

What it solves

This project provides a streamlined, one-command deployment solution for running the GLM-5.3-Flash large language model across multiple NVIDIA DGX Spark units. It addresses the complexity of setting up high-performance inference clusters by automating the orchestration of model weights, speculative decoding, and network configuration.

How it works

It uses a containerized approach to deploy an OpenAI-compatible vLLM server. The system handles the distribution of model weights across connected Spark units, implements DFlash2 speculative decoding to improve token generation speed, and supports various performance-tuning profiles (such as "Speed boost" and "Abliteration") through environment variable configuration. It is designed specifically for the GB10 architecture.

Who it’s for

This project is for engineers and researchers who have access to NVIDIA DGX Spark hardware and need to serve the GLM-5.3-Flash model with high throughput and low latency. It is intended for users comfortable with Linux-based cluster management and environment-level configuration.

Highlights

  • Supports multi-node scaling across 2, 3, or 4 DGX Spark units.
  • Includes DFlash2 speculative decoding for faster token generation.
  • Provides an OpenAI-compatible API for easy integration with existing LLM applications.
  • Offers opt-in performance profiles for specialized needs like uncensored output (Abliteration) or optimized decoding speeds.
  • Features extensive configuration options for context length, request concurrency, and hardware-specific tuning.

Related

  • Project
    MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks-TensorFold

    This project provides a ready-to-run recipe for serving the GLM-5.3-Flash language model across two NVIDIA DGX Sparks using TensorFold, delivering an OpenAI-compatible API with a 1M-token context, image/video input, and high throughput at low latency.

  • Project
    eugr/spark-vllm-docker

    A collection of Docker configurations and automation scripts for deploying high-performance vLLM inference on single or multi-node NVIDIA DGX Spark clusters.

  • Project
    MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark

    A deployment recipe for running DeepSeek-V4-Flash-Vision-Exp on two-node DGX Spark clusters, featuring 1M-token context support, speculative decoding, and native image processing.

  • Project
    MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark

    A deployment recipe for serving the Qwen3.8-Flash-Next vision-language model on a single DGX Spark using vLLM, with memory optimizations, speculative decoding, and extensive benchmark data.

  • Dispatch
    vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation

    vLLM provides a high-performance local inference endpoint for NVIDIA DGX Spark, enabling the deployment of large NVFP4 models like Nemotron-3-Super using a unified-memory architecture and OpenAI-compatible API.