NVIDIA DGX Spark and Reachy Mini Integration Guide

At CES 2026, NVIDIA unveiled a system to bring AI agents into the physical world by combining the processing power of DGX Spark with the Reachy Mini robot. This integration allows developers to build a private, customizable office assistant capable of real-time voice and vision interaction, powered by open reasoning and vision models.

Core Technical Components

The system is built on a modular stack of open models and frameworks, allowing developers to swap components without rewriting core logic. The primary "ingredients" include:

  • Reasoning Model: NVIDIA Nemotron 3 Nano (30B A3B BF16).
  • Vision Model: NVIDIA Nemotron Nano 2 VL (12B v2 VL BF16).
  • Text-to-Speech: ElevenLabs.
  • Hardware/Simulation: Reachy Mini (physical hardware or simulation).
  • Orchestration: NVIDIA NeMo Agent Toolkit.
  • Real-time Multimodal Handling: Pipecat framework for low-latency audio/video streams.

Agent Architecture and Orchestration

The NeMo Agent Toolkit serves as the central hub, connecting the models to the hardware. It is designed to be framework-agnostic and compatible with other agentic frameworks such as LangChain, LangGraph, and CrewAI.

Intent-Based Routing

To optimize for latency and cost, the system does not use a single model for all tasks. Instead, it employs a router (such as the microsoft/phi-3-mini-128k-instruct model) to direct queries based on intent:

  • Text Queries: Routed to a fast text model for simple chit-chat and casual conversation.
  • Visual Queries: Routed to a Vision Language Model (VLM) when the user asks about their environment, appearance, or objects in view.
  • Action/Tool Requests: Routed to a ReAct agent for tasks requiring external information or physical robot actuation.

Tool Calling with ReAct

The system utilizes a built-in ReAct agent within the NeMo Agent Toolkit to handle tool calling. This allows the agent to reason between multiple tool calls—such as performing a Wikipedia search or triggering a specific robot behavior—before providing a final answer. To ensure safety and stability, the implementation recommends tight tool schemas and a hard cap on the maximum number of tool calls (max_tool_calls).

Real-Time Interaction and Hardware Integration

Real-time capabilities are managed by Pipecat, which orchestrates the robot's camera for vision, speech recognition for input, and text-to-speech for output.

The Reachy Mini robot acts as the physical endpoint. It exposes a daemon that the bot service connects to, allowing the same Python code to control either a physical robot or a simulation. This architecture ensures that perception, reasoning, and action are tightly coupled in a physical form factor.

Deployment Options

Developers can deploy this agent stack through three primary methods:

  1. Local Deployment: Running on DGX Spark or GPUs with sufficient VRAM (requiring approximately 65GB disk space for the reasoning model and 28GB for the vision model).
  2. Cloud Deployment: Utilizing cloud GPUs via NVIDIA Brev or Hugging Face Inference Endpoints.
  3. Serverless Endpoints: Connecting to remote endpoints via NVIDIA or Hugging Face Inference Providers.

Sources