Arm Neural Super Sampling (NSS) Release

Arm has released Neural Super Sampling (NSS), an AI-powered upscaling solution that enables high-resolution rendering at a lower compute cost for mobile devices. This technology is designed to optimize performance in power-constrained environments such as mobile gaming and Extended Reality (XR).

High-Resolution Rendering with Reduced GPU Load

Neural Super Sampling (NSS) reduces the computational burden on the GPU by reconstructing high-quality output frames from low-resolution temporal inputs. In a demonstrated "Enchanted Castle" demo, NSS reduced the GPU workload by 50 percent, rendering at 540p and upscaling to 1080p with a latency of 4ms in a sustained performance setup.

Technical Architecture and Optimization

NSS is a parameter prediction model for real-time temporal super sampling. It is specifically optimized for execution on Neural Accelerators (NX) within mobile GPUs, utilizing Arm Neural Technology to achieve real-time performance on future mobile hardware.

Training Data and Model Development

To support the development of NSS, Arm has provided the Neural Graphics Dataset, a collection of reference images, image sequences, and associated motion and depth data. While the current dataset is a limited set intended to demonstrate the model development flow, Arm plans to release tools via the Neural Graphics Model Gym to allow developers to capture and convert content for full model training and retraining.

Integration and Developer Tools

NSS is available for experimentation through integration with Unreal Engine via two specific plugins:

• NSS Plugin for Unreal Engine: Dedicated plugin for NSS integration.
• Unreal NNE Plugin for ML extensions for Vulkan: Provides support for ML extensions for Vulkan.

Developers can access the NSS model on Hugging Face and utilize Arm's provided quickstart guides for Vulkan ML samples and Unreal Engine integration.