spark-arena/sparkrun
sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems
What it solves
It simplifies the deployment and management of LLM inference workloads on NVIDIA DGX Spark systems. It removes the need for complex orchestrators like Slurm or Kubernetes, allowing users to launch models across one or more nodes with a single command.
How it works
Sparkrun uses a guided setup wizard to configure a cluster, including SSH mesh, sudoers, and network interface detection. It manages model and container distribution across nodes via SSH and supports multi-node tensor parallelism by automatically detecting InfiniBand/RDMA. It utilizes "recipes" from Git-based registries to define how models should be run.
Who it’s for
Developers and researchers working with NVIDIA DGX Spark hardware who want a streamlined way to deploy LLM inference without the overhead of traditional cluster management tools.
Highlights
- Multi-runtime support: Works with vLLM, SGLang, and llama.cpp.
- Multi-node tensor parallelism: Easily scale across multiple hosts with automatic network detection.
- VRAM estimation: Predicts if a model will fit in memory before launching.
- Git-based recipe registries: Access to official, community, and benchmarked model configurations.
- Automated setup: Guided wizard for cluster creation, SSH mesh, and hardware detection.