slhck/ffmpeg-quality-metrics

Calculate quality metrics with FFmpeg (SSIM, PSNR, VMAF, VIF)

What it solves

When you compress or transcode a video, you often need to know how much quality was lost compared to the original. This tool computes standard video quality metrics — PSNR, SSIM, VIF, MSAD, and VMAF — by comparing a distorted (encoded) video against a reference (original) one. It saves you from manually wiring up FFmpeg commands and parsing raw output, giving you per-frame numbers, global statistics (min/max/average/standard deviation), and an optional interactive browser dashboard.

How it works

You give it two video files: a distorted version and a reference (e.g., ffmpeg-quality-metrics distorted.mp4 reference.y4m). The tool uses FFmpeg under the hood to decode both streams, automatically scales the distorted video to the reference's resolution, and applies the requested metric filters. It first probes both inputs with ffprobe to warn about differences in resolution, frame rate, or duration. Then it runs FFmpeg and collects the per-frame metric values. By default it computes PSNR and SSIM; you can enable other metrics with the --metrics option. Output is JSON or CSV, and you can also launch a web-based GUI to visually explore the results.

For VMAF, it supports selecting different model files (including the newer v1 models) and passing extra parameters like feature flags or 10-bit conversion. You can also limit the analysis to a specific number of frames or start from a certain timestamp or frame number. The tool handles misalignment: use --dist-delay to trim unmatched lead-in frames so both videos are time-synced before comparison.

Who it's for

Video engineers, quality assurance teams, researchers, or anyone who encodes video and needs objective quality scores. It's a command-line tool that requires Python 3.10+ and FFmpeg 7.1+ (with libvmaf if you want VMAF). It works on Linux, macOS, and Windows and can be installed via pip, pipx, uvx, or run via Docker.

Highlights

  • Supports five metrics: PSNR, SSIM, VIF, MSAD, and VMAF.
  • Outputs per-frame metrics and global statistics (min/max/avg/stddev).
  • Optional browser-based interactive dashboard for visual analysis.
  • Automatically scales and aligns inputs, with detailed warnings about mismatches.
  • Flexible control: choose metrics, model files, frame ranges, and start offsets.
  • Can output JSON or CSV, and writes to stdout or a file.
  • Works as a CLI or as a Python library (API).

Related

  • Project
    PacificAI/langtest

    A comprehensive evaluation library for NLP and LLM models to test for robustness, bias, fairness, and safety.

  • Project
    scaleapi/SWE-bench_Pro-os

    A challenging benchmark for evaluating LLMs and AI agents on long-horizon software engineering tasks by requiring them to generate patches for real-world codebase issues.

  • Project
    zli12321/LHTB

    LHTB is a benchmark that tests how well LLM agents can keep working in a containerised terminal over hundreds of steps. It provides 77 diverse, stateful tasks, a patched Harbor harness that enforces a continue‑until‑timeout policy, hidden verifiers, and published leaderboards showing model performance and cost.

  • Project
    OWASP/AISVS

    A community‑driven, testable checklist (AISVS) for securing AI systems across their lifecycle, offering three verification levels and detailed chapters on data integrity, model management, adversarial robustness, and more.