NVIDIA-RTX/RTXNS

What it solves

RTX Neural Shading (RTXNS) provides a starting point for developers to integrate machine learning into graphics applications. It simplifies the process of training neural networks to replace traditional graphics rendering components, such as shaders or textures, and then running those models for real-time inference on the GPU.

How it works

The project utilizes the Slang shading language and leverages GPU ML acceleration via either the DirectX Preview Agility SDK or the Vulkan Cooperative Vectors extension. It includes a set of progressive examples that guide developers through training and deploying neural networks for shading tasks. Additionally, it integrates with SlangPy, allowing developers to use Python for faster iteration and development of neural networks before integrating them into the final application.

Who it’s for

Graphics developers who are already familiar with machine learning and neural networks and want to bring ML-driven rendering techniques to Windows or Linux applications.

Highlights

• ML-Driven Rendering: Examples showing how to train neural networks to represent shaders or textures.
• Cross-Platform Support: Works on Windows and Linux using DirectX or Vulkan.
• Fast Iteration: Integration with SlangPy for Python-based neural network development.
• GPU Acceleration: Utilizes specialized hardware acceleration through the Vulkan VK_NV_cooperative_vector extension and DirectX Preview SDKs.