nv-tlabs/ardy

Official implementation of ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation (SIGGRAPH 2026).

What it solves

ARDY (Autoregressive Diffusion with Hybrid Representation) solves the challenge of generating realistic, interactive human motion that responds in real-time to both text prompts and specific spatial constraints. It allows users to guide character movement using natural language while simultaneously enforcing precise kinematic requirements like foot placement, root trajectories, or specific body poses.

How it works

ARDY uses an autoregressive diffusion model that processes motion in continuous, real-time streams. It combines a large language model (via LLM2Vec) to interpret text prompts with a motion-generation engine that can be constrained by external inputs. The system supports "online" generation, meaning it can adjust the character's movement on the fly as a user provides new waypoints, velocity targets, or keyframe constraints. It includes a browser-based interactive demo and a command-line interface for batch processing.

Who it’s for

This project is for researchers and developers working on character animation, robotics simulation, or interactive virtual environments who need a high-fidelity, controllable motion synthesis system.

Highlights

  • Real-time interactive control via mouse, keyboard, and streaming text prompts.
  • Flexible constraint support including root paths, waypoints, full-body keyframes, and end-effector positions.
  • Supports multiple skeleton types, including humanoids and specific robotic platforms like the Unitree G1.
  • Includes an interactive web-based GUI for live motion visualization and control.
  • Provides options for TensorRT acceleration to ensure low-latency performance.

Related

  • Project
    intel/xess

    The XeSS 3 SDK is a set of AI-based technologies for real-time rendering that boosts frame rates and reduces input lag through super resolution and frame generation.

  • Project
    markaren/threepp

    A cross-platform C++20 3D library that ports the three.js API to C++, featuring advanced Vulkan rendering, PhysX robotics simulation, and synthetic data generation.

  • Project
    jayin92/Skyfall-GS

    Skyfall-GS is a framework that synthesizes immersive, large-scale 3D urban scenes from satellite imagery and diffusion models without requiring costly 3D annotations.

  • Project
    Bingeljell/image-to-3dlab

    AssetFurnace is a local, web‑based studio that converts a text prompt or 2‑D image into a rigged, animated, game‑ready 3‑D GLB asset. It supports multiple generation back‑ends (Pixal3D, Hunyuan‑3D, TRELLIS.2, Stable Fast 3D), runs on Apple Silicon, NVIDIA, or AMD GPUs, and includes a Pixel‑Match step that projects the original image’s pixels onto the mesh for crisp details. The tool is installed with a single script, works entirely offline, and records licence provenance for each output.