WenjieDu/SAITS

The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516

What it solves

SAITS is designed to fill in missing values in multivariate time series (imputation). It specifically addresses the limitations of RNN-based models, such as slow training speeds, memory constraints, and the accumulation of errors over time (compounding error).

How it works

SAITS uses a pure self-attention mechanism without any recursive design. It employs a joint-optimization training approach to improve accuracy and efficiency. The framework is flexible enough that its embedding and training strategies have been adapted to over 20 other forecasting models to enable them to perform time-series imputation.

Who it’s for

Researchers and data scientists working with partially-observed time series datasets who need a high-accuracy, efficient imputation method for sequence data.

Highlights

  • High Performance: Outperforms BRITS in MAE by 12% to 38% and is 2.0 to 2.6 times faster to train.
  • Efficient Architecture: Uses significantly fewer parameters (15% to 30% of a standard Transformer) while maintaining or exceeding its performance.
  • Non-Recursive: The first work to apply pure self-attention for general time series imputation without recursive designs.
  • Toolbox Integration: Now available as part of the PyPOTS toolbox for easier implementation.

Related

  • Project
    WenjieDu/PyPOTS

    A Python toolbox for machine learning on partially-observed time series, providing unified APIs for imputation, forecasting, and anomaly detection on data with missing values.

  • Project
    Nixtla/neuralforecast

    A library of over 30 state-of-the-art neural forecasting models designed for high performance, usability, and efficiency in time-series forecasting.

  • Project
    Nixtla/statsforecast

    A high-performance library for univariate time series forecasting that provides fast, scalable implementations of statistical and econometric models.

  • Project
    e-sensing/sits

    SITS is an open‑source R package (with a Python wrapper) for building, regularising, and classifying satellite‑image time‑series data cubes. It connects to many public EO collections, offers a suite of ML/DL models (including GPU‑accelerated CNNs and attention encoders), and provides end‑to‑end tools for training, classification, smoothing, uncertainty estimation, and active learning.

  • Project
    timeseriesAI/tsai

    A state-of-the-art deep learning library built on PyTorch and fastai for time series tasks including classification, regression, and forecasting.