FFmpeg 9.0 Release Notes

FFmpeg 9.0 "Lei" is a major release that expands hardware acceleration support, introduces new multimedia formats, and refines deep learning integration. This release arrives approximately four months after FFmpeg 8.1.

Hardware Acceleration and GPU Enhancements

FFmpeg 9.0 significantly expands its hardware-accelerated processing capabilities, specifically targeting AMF, Vulkan, and CUDA.

AMF (Advanced Media Framework) Updates

  • HDR Capabilities: The AMF Color Converter (vf_vpp_amf) now has extended HDR capabilities.
  • Frame Rate Conversion: A new AMF Frame Rate Converter filter (vf_frc_amf) has been added.
  • Memory Management: The release introduces support for AMF hardware memory mapping.

Vulkan and CUDA Integration

  • Vulkan Filters: The v360_vulkan filter is now available for spherical video processing.
  • APV Support: Added Vulkan hardware acceleration for APV.
  • CUDA Filters: The transpose_cuda filter has been added to enable GPU-accelerated image rotation and flipping.

Other Hardware Accel

  • ProRes RAW: Added VideoToolbox hardware acceleration for ProRes RAW.

New Codecs and Format Support

This version adds support for several niche and modern formats to ensure broad compatibility across different device ecosystems.

  • Animated WebP: Added both a decoder and a demuxer for Animated WebP files.
  • Playdate: Introduced a dedicated video encoder and muxer for the Playdate handheld console.
  • HE-AAC 960: Added decoding support for HE-AAC 960 (used in DAB+).
  • MP4 Muxer: Added support for LCEVC track muxing within the MP4 container.
  • Dolby Vision: A new bitstream filter has been implemented to split multi-layer HEVC for Dolby Vision.
  • Metadata: Added support and passthrough for SMPTE 2094-50 metadata.

Deep Learning and Backend Changes

FFmpeg 9.0 improves its integration with AI-driven processing through the ONNX Runtime.

  • ONNX Runtime: The DNN backend now supports the GPU execution provider, allowing deep learning filters to run on hardware accelerators rather than relying solely on the CPU.

Deprecations and Removals

To maintain codebase health, FFmpeg 9.0 removes several legacy components:

  • CELT: Removed decoding support for CELT and ogg/celt parsing (this does not affect Opus CELT).
  • NVENC: Removed deprecated NVENC options and support for SDK versions prior to 11.1.

Community Insights and Technical Discussion

Users and developers on Hacker News highlighted the enduring importance of FFmpeg as a foundational piece of internet infrastructure. Some technical points and critiques emerged from the discussion:

  • Filter Complexity: Some users find the FFmpeg filter system "confusing to use" and "not elegant," suggesting a need for a higher-level meta-language or DSL to treat multimedia data as a tangible object-oriented stream.
  • Platform Limitations: There are ongoing requests for better Intel QSV encoding support on Windows laptops where manufacturers have disabled capabilities in ACPI tables, a feature currently only functional on Linux.
  • Performance Philosophy: The community noted the project's commitment to extreme efficiency, with engineers still "hand-rolling assembler code to squeeze out another 5% efficiency."

"Watching ffmpeg go from a niche tool for web back ends to what it is now has been incredible."

"I'd wish we could use some kind of simple meta-language... to think of multimedia data as tangible to manipulation as if it were an object oriented system or datastream system."

Sources