thedatumorg/TSB-AD

What it solves

TSB-AD addresses the "elephant in the room" regarding time-series anomaly detection (TSAD): flawed datasets, biased evaluation measures, and inconsistent benchmarking practices. It provides a standardized framework to reliably compare different detection algorithms across diverse data types.

How it works

TSB-AD implements a comprehensive benchmarking suite consisting of three core components:

1. Dataset Integrity: A curated collection of 1,070 high-quality time series from 40 diverse datasets, combining human perception and model interpretation.
2. Measure Reliability: It identifies and utilizes VUS-PR as the most reliable and accurate evaluation measure to avoid common biases in TSAD metrics.
3. Comprehensive Benchmarking: It integrates 40 different detection algorithms—ranging from traditional statistical methods (like Isolation Forest and PCA) to neural networks (like LSTM and Transformers) and modern foundation models (like Chronos and TimesFM)—with unified setups and hyperparameter tuning for fair comparison.

Who it’s for

Researchers and practitioners in data science and machine learning who need to evaluate and compare the performance of time-series anomaly detection models on real-world, heterogeneous data.

Highlights

• Large-scale curated data: Includes univariate (TSB-AD-U) and multivariate (TSB-AD-M) datasets.
• Broad algorithm spectrum: Supports 40 algorithms across statistical, neural network, and foundation model categories.
• Standardized evaluation: Uses the VUS-PR metric to ensure reliable accuracy measurements.
• Reproducible results: Provides a unified setup and hyperparameter tuning scripts to ensure fair comparisons.