decisionintelligence/TFB

[PVLDB 2024 Best Paper Nomination] TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods

What it solves

TFB addresses the lack of fair and comprehensive benchmarking in time series forecasting. It prevents common evaluation pitfalls—such as the "drop-last" trick where incomplete batches are discarded during testing—which can lead to inconsistent results and unfair comparisons between different forecasting models.

How it works

TFB provides a standardized end-to-end pipeline for evaluating time series forecasting models. It includes a clean codebase for training and testing, a collection of pre-processed datasets, and a wide array of baseline algorithms (such as DLinear, PatchTST, and DUET). The library also includes a characteristics extractor to compute dataset properties like trend, seasonality, and stationarity.

Who it’s for

It is primarily designed for researchers in time series forecasting who need a reliable framework to develop, test, and compare their new methods against established baselines using consistent evaluation strategies.

Highlights

  • Fair Evaluation: Eliminates the drop-last operation during testing to ensure consistent test set usage across all models.
  • Extensive Baselines: Integrates numerous state-of-the-art forecasting models including TimeKAN, xPatch, and HDMixer.
  • Comprehensive Tooling: Includes tools for computing time series characteristics and support for predicting specific subsets of input variables.
  • Ready-to-use Data: Provides access to 27 multivariate datasets for immediate benchmarking.

Related

  • Project
    AIStream-Peelout/flow-forecast

    An open-source deep learning framework for time series forecasting, classification, and anomaly detection featuring a wide array of state-of-the-art models.

  • Project
    skforecast/skforecast

    A Python library for time series forecasting that integrates scikit-learn compatible models, statistical methods, and foundation models into a unified API.

  • Project
    LongxingTan/Time-series-prediction

    TFTS is a TensorFlow-based library that provides a unified interface for implementing state-of-the-art deep learning models for time series prediction, classification, and anomaly detection.

  • Project
    PriorLabs/tabpfn-time-series

    A zero-shot time series forecasting library that frames forecasting as a tabular regression problem using the TabPFN foundation model.

  • Project
    mikekeith52/scalecast

    A time series forecasting library that provides a uniform interface for ML and deep learning models, simplifying model selection, validation, and pipeline creation.