NixOS-DGX-Spark: Running Nix and NixOS on NVIDIA DGX Spark and Asus Ascent GX10

Overview

The NixOS-DGX-Spark repository offers a complete Nix‑based solution for NVIDIA DGX Spark and the compatible Asus Ascent GX10 devices. It supplies a NixOS module, pre‑built USB images, and devshell playbooks that let users either run Nix on the existing DGX OS (Ubuntu) or replace the system with a full NixOS installation.

Using Nix on DGX OS (Ubuntu)

You can use the repository’s dev shells and playbooks on DGX OS without installing NixOS. The setup requires installing Nix (via the official or Determinate installer), enabling the nix-command and flakes experimental features in /etc/nix/nix.conf, and adding the graham33 Cachix cache as described in the repository’s Caching section. Once configured, commands such as nix develop .#cuda automatically wrap GPU‑dependent binaries with nixglhost from the numtide/nix-gl-host project, so no manual driver handling is needed.

Installing NixOS on DGX Spark

A warning in the repository states that only the factory DGX OS firmware can boot the device out of the box; to run NixOS you must first update the firmware from within DGX OS. After that step, you can build a bootable USB image with nix build .#usb-image and write it to a USB drive using dd. The image presents two kernel choices in GRUB:

  • NixOS (default) – uses NVIDIA’s custom kernel for the DGX Spark, providing full GPU support and working Ethernet.
  • NixOS (standard-kernel) – uses the generic NixOS 6.17 kernel, which the documentation notes has Ethernet problems.

Booting requires disabling Secure Boot in the BIOS and then following the standard NixOS installation manual.

DGX Spark NixOS Module

The repository’s NixOS module (hardware.dgx-spark) encapsulates hardware support. Its core options are:

hardware.dgx-spark = {
  enable = true;                 # enable the module
  useNvidiaKernel = true;        # default: use NVIDIA‑optimized kernel
};

When useNvidiaKernel = true the module pulls in a kernel built from NVIDIA’s Debian annotations, configured to differ only where necessary from the NixOS default—a process that reduces the configuration size by roughly 82 %. The module also enables the DGX Dashboard web interface at http://localhost:11000 for GPU telemetry and system monitoring.

If you prefer the standard NixOS kernel (despite its networking caveats), set useNvidiaKernel = false.

Kernel Configuration Management

The kernel configuration files live in kernel-configs/nvidia-dgx-spark-<version>.nix. They are generated by comparing NVIDIA’s annotated kernel source with the NixOS baseline, keeping only the differing options. To regenerate them after a kernel or nixpkgs update, run:

nix run .#generate-kernel-config

The command fetches the NVIDIA kernel source, builds the NixOS default config, compares the two, and writes a terse diff‑based config. Regeneration is required when the NVIDIA kernel version changes (update kernel-configs/nvidia-kernel-source.nix) or when the common NixOS kernel configuration changes in nixpkgs.

Importing the Module

Other flakes can consume the module via:

{
  inputs.dgx-spark.url = "github:graham33/nixos-dgx-spark\);ings = { nixpkgs, dgx-spark, ... }:
    nixosConfigurations.mySystem = nixpkgs.lib.nixosSystem {
      modules = [
        dgx-spark.nixosModules.dgx-spark
        { hardware.dgx-spark.enable = true; }
        # additional modules
      ];
    };
}

A quick‑start template is also provided: nix flake init -t github:graham33/nixos-dgx-spark#dgx-spark creates a directory with flake.nix, configuration.nix, and a hardware‑configuration template that you can customize before copying to /etc/nixos and applying with sudo nixos-rebuild switch --flake /etc/nixos#dgx-spark.

Playbooks

The repository bundles a set of devshell‑based playbooks derived from NVIDIA’s DGX Spark examples. Each playbook is labeled by its reproducibility level:

  • Full Nix² – all dependencies are obtained via Nix, no containers.
  • Container³ – Nix supplies tooling (e.g., podman) but containers are pulled/built at runtime.
  • Nix + Container⁴ – CLI tools are Nix‑packaged, while the tool itself manages containers.

The table in the README lists playbooks such as ComfyUI, Connect Two Sparks, DGX Dashboard, FLUX.1 Dreambooth, Multi‑Agent Chatbot, Multi‑modal Inference, NCCL for Two Sparks, NVFP4, OpenShell, PyTorch fine‑tuning (both container and native Nix), Speculative Decoding, TRT‑LLM, and vLLM (container and native). Checkmarks indicate which playbooks have been tested on NixOS and/or DGX OS; for example, the ComfyUI and DGX Dashboard playbooks are marked as working on both platforms, while the NVFP4 and OpenShell playbooks have only been tested on NixOS.

Caching

To avoid rebuilding large CUDA stacks, the project recommends two binary caches:

  1. Flox CUDA cache – provides aarch64‑linux builds of cudatoolkit, nccl, cuDNN, PyTorch, etc., with NVIDIA’s permission. When using the DGX Spark NixOS module, the Flox cache is added automatically as a substituter. For standalone Nix on DGX OS, add to /etc/nix/nix.conf:
    extra-substituters = https://cache.flox.dev
extra-trusted-public-keys = flox-cache-public-1:7F4OyH7ZCnFhcze3fJdfyXYLQw/aV7GEed86nQ7IsOs=
  2. graham33 Cachix cache – hosts artifacts built specifically for this repository (e.g., dgx-dashboard, openshell). Use it with cachix use graham33 after installing the cachix tool.

Firmware Updates

Because the factory firmware only boots DGX OS, the repository advises updating firmware from within DGX OS before attempting to boot NixOS. The module enables fwupd, allowing you to check and apply updates via:

fwupdmgr get-updates
fwupdmgr update

These commands contact the Linux Vendor Firmware Service (LVFS) where NVIDIA publishes DGX Spark firmware.

Experimental Remote Installation

An experimental section describes using nixos-anywhere to install NixOS over SSH. The command nix run github:nix-community/nixos-anywhere -- --flake github:graham33/nixos-dgx-spark#dgx-spark root@<ip> partitions the NVMe disk and installs the system with the DGX Spark module enabled. A VM‑test mode is available for validation.

Community Feedback

Comments on the Hacker News post highlight early adopters’ experiences:

  • "Huge plus to anyone interested in the space to check out what Graham built!" – @ronef

  • "This is incredible. I have a Jetson lying around and will try to it out on this." – @thenobsta

  • "Been running this on a few Asus GX10 machines with k3s on top, it’s been great. I’m running the new deepseek." – @redrove

  • "Thanks for sharing, saving this for when I get a DGX Spark" – @haunter

  • "This has been amazingly helpful for managing my DGX Spark! Thank you for all your time and effort into this project!" – @nixie-tubes

  • "Claude Code (and likely other models/harnesses) are incredibly effective at Nix. It can trivially self‑verify, without side effects, which is a perfect match for an LLM." – @hamandcheese

  • "There is also a microvm.nix project which helped us support sandboxes with firecracker. So, whole ai workflow pipeline can now be nixos." – @mkagenius

These remarks reflect enthusiasm for using Nix on edge AI hardware, interest in combining the project with Kubernetes (k3s) on the Asus GX10, and appreciation for the reproducibility and self‑verification properties that Nix brings to AI workloads.

License

The project is released under the MIT License; see the included LICENSE file for details.

Sources