NVIDIA-RTX/RTXNTC

NVIDIA Neural Texture Compression SDK

What it solves

NTC solves the problem of high memory consumption and bandwidth usage for PBR (Physically Based Rendering) material textures. Instead of compressing each texture channel (like albedo, normal, roughness) separately using traditional block compression, NTC compresses multiple correlated texture channels together into a single neural texture set, significantly reducing the VRAM footprint and PCI-E traffic.

How it works

During compression, texture data is transformed into weights for a small neural network (a Multi-Layer Perceptron or MLP decoder) and a tensor of latents. To reconstruct the texture, the system samples these latents and passes them through the decoder.

Depending on the hardware and performance needs, it supports three decompression modes:

  • Inference on Load: Textures are decompressed and transcoded to standard block-compressed formats (BCn) when the game or map loads.
  • Inference on Sample: Textures are decompressed in real-time directly within shaders (pixel or ray tracing hit shaders).
  • Inference on Feedback: Uses Sampler Feedback to decompress only the specific texture tiles needed for the current view.

To accelerate this process, it leverages "Cooperative Vector" extensions (Vulkan VK_NV_cooperative_vector and DX12 LinAlg) on Ada- and Blackwell-class GPUs.

Who it’s for

This SDK is designed for graphics engineers and game developers who want to reduce the memory footprint of high-quality materials in 3D rendering pipelines while maintaining visual quality comparable to standard block compression.

Highlights

  • Multi-channel Compression: Can compress up to 16 texture channels into one set.
  • Adjustable Quality: Offers a constant bitrate scheme where users can select "Latent Shapes" to balance memory usage and image quality.
  • Hardware Acceleration: Utilizes Cooperative Vector extensions for 2-4x faster inference on modern NVIDIA GPUs.
  • Flexible Integration: Includes a C++ library (LibNTC), a command-line tool (ntc-cli), and a Python module for automation.

Related

  • Project
    microsoft/DCVC

    An ultra-fast neural video codec that uses a chunk-based coding framework to achieve high compression efficiency and rapid encoding/decoding speeds.

  • Project
    openvinotoolkit/nncf

    NNCF is a Python library for compressing neural networks (quantization, weight compression, pruning) that works with PyTorch, TorchFX, ONNX and OpenVINO. It provides both post‑training and training‑time algorithms, easy‑to‑use APIs, GPU‑accelerated fine‑tuning, and integration with OpenVINO for fast, low‑memory inference on edge devices.

  • Project
    nv-tlabs/NKSR

    Neural Kernel Surface Reconstruction (NKSR) is a method for creating high-quality 3D implicit surfaces from large-scale, sparse, and noisy point clouds, capable of handling millions of points in seconds.

  • Project
    CERN/TIGRE

    A GPU-accelerated toolbox for fast and accurate 3D tomographic reconstruction using Python and MATLAB, supporting a wide range of iterative algorithms and flexible CT geometries.

  • Project
    NVIDIA-RTX/NRD

    A spatio-temporal denoising library for real-time path tracing that resolves noise on opaque surfaces using G-buffer guides across D3D12, Vulkan, and D3D11.