SciML/SciMLBenchmarks.jl

What it solves

SciMLBenchmarks.jl provides a standardized, open-source framework for evaluating the performance of scientific machine learning (SciML) tools and equation solvers. It addresses the need for fair, reproducible benchmarks that compare the speed, robustness, and accuracy of various numerical methods across different languages and implementations.

How it works

The project maintains a collection of benchmark suites—including notebooks and Weave.jl files—that test solvers against a wide range of problems. It measures efficiency using "work-precision diagrams," which plot timing against error tolerances to determine the most efficient method for a given accuracy level. The benchmarks are executed on standardized hardware (AMD EPYC processors) to ensure reproducibility and are automatically generated into PDFs, HTML, and notebooks.

Who it’s for

Researchers, developers, and practitioners in computational science and AI for science who need to compare equation solvers, parameter estimation methods, or neural operators to choose the best tool for their specific mathematical problem.

Highlights

• Comprehensive Coverage: Benchmarks a wide array of methods including non-stiff/stiff ODEs, SDEs, DDEs, and physics-informed neural networks (PINNs).
• AI for Science Integration: Includes comparisons of surrogate models like DeepONets and Fourier Neural Operators.
• Reproducible Infrastructure: Uses self-hosted GitHub Actions runners with specific CPU and GPU configurations to maintain consistent hardware baselines.
• Interactive Exploration: Allows users to generate and run interactive notebooks locally to verify or extend benchmark results.