PyTorch 2.15 MPS Release Notes

Overview

PyTorch 2.15 significantly improves performance and memory efficiency for Apple Silicon users by introducing native FP8 support, optimized LLM decoding paths, and a shift toward dedicated Metal kernels. These updates reduce reliance on MPSGraph, resulting in faster inference for dense and Mixture-of-Experts (MoE) models, as well as more stable memory usage for complex workloads like video generation.

Key Technical Improvements

FP8 Support

PyTorch 2.15 adds support for torch.float8_e4m3fn tensors on the MPS backend. This allows developers to store values using only one byte per element, effectively halving the memory footprint compared to FP16 or BF16. The release includes F.scaled_mm for direct FP8 matrix multiplication on the GPU, supporting both tensorwise and rowwise scaling.

LLM Decoding Optimization

Decoding performance has been substantially improved for both dense and MoE models by bypassing MPSGraph in favor of native Metal kernels:

  • Linear Layers: F.linear now utilizes specialized matrix-vector kernels, providing a 1.23× geometric-mean speedup across common decoding shapes.
  • MoE Models: A new native grouped matmul implementation allows for faster token routing. On an M3 Ultra using torch.compile, MoE models like Qwen3.6-35B-A3B saw throughput increase from 9.4 to 61.3 tokens/s.
  • Softmax: Dedicated Metal kernels have reduced latency for softmax and log_softmax operations by approximately 8× compared to previous MPSGraph implementations.

Memory Management in Video Workloads

By replacing MPSGraph-based padding with dedicated Metal kernels, PyTorch 2.15 resolves significant memory overhead issues in video processing. For example, the tiled VAE decoding process for Wan2.2 now peaks at 8 GiB of memory, down from 270 GiB in previous versions.

Linear Algebra and Operations

Linear algebra capabilities have been expanded to include:

  • Complex-valued LU factorization, LU solve, and triangular solve.
  • Optimized kernels for small-matrix inversion (up to 4x4) and improved robustness for SVD operations.
  • New Metal-based paths for conv1d, constant/reflection/replication padding, and upsampling backward.

Developer Control and Requirements

Developers now have more granular control over execution via torch.mps.Stream and the torch.mps.stream context manager, allowing for explicit stream management and synchronization.

System Requirements:

  • macOS: 15 or later.
  • Python: 3.11 or later.

Future Outlook: PyTorch 2.16

Development for the 2.16 release is focused on further reducing MPSGraph dependency. Planned improvements include:

  • Full Metal Attention: Transitioning scaled dot-product attention entirely to Metal kernels.
  • Command Buffer Optimization: Committing command buffers every 16 kernels to allow concurrent GPU/CPU execution during decoding.
  • Binary Size Reduction: Compiling bundled shaders for Metal 3.1 by default to reduce the size of libtorch_cpu.dylib.

Sources