PatchTSMixer added to Hugging Face Transformers – release and quick‑start guide

TL;DR

PatchTSMixer, a lightweight MLP‑Mixer‑based time‑series model from IBM Research, is now available in the Hugging Face Transformers library, enabling fast, memory‑efficient forecasting, classification, and regression with state‑of‑the‑art accuracy.

What is PatchTSMixer?

PatchTSMixer splits multivariate time‑series inputs into fixed‑size patches, embeds them, and processes the resulting tensor with a stack of MLP‑Mixer layers that learn inter‑patch, intra‑patch, and inter‑channel correlations. Residual connections and gated attention modules help the model focus on salient features. The architecture supports both masked pre‑training and direct forecasting, and can be configured with various attention blocks.

Performance Claims

According to the original IBM Research paper, PatchTSMixer improves forecasting accuracy over existing MLP and Transformer baselines by 8 %–60 % and beats recent Patch‑Transformer models by 1 %–2 %, while using 2 ×–3 × less memory and runtime.

Getting Started: Installation

To use PatchTSMixer you need two Python packages:

pip install git+https://github.com/IBM/tsfm.git   # IBM Time Series Foundation Model repository
pip install transformers                         # Hugging Face Transformers

A quick sanity check:

from transformers import PatchTSMixerConfig
from tsfm_public.toolkit.dataset import ForecastDFDataset

If the imports succeed, the environment is ready.

Example 1 – Direct Forecasting on the Electricity Dataset

The notebook demonstrates a full training pipeline:

  1. Seed setting – reproducibility with set_seed(42).
  2. Data loading – CSV is read with pandas; columns are split into training/validation/test based on index ranges.
  3. Pre‑processing – TimeSeriesPreprocessor scales each window (default "std") and creates sliding context windows of length 512.
  4. Model configuration – Example hyper‑parameters:
    config = PatchTSMixerConfig(
    context_length=512,
    prediction_length=96,
    patch_length=8,
    patch_stride=8,
    num_input_channels=...,
    d_model=16,
    num_layers=8,
    expansion_factor=2,
    dropout=0.2,
    head_dropout=0.2,
    mode="common_channel",
    scaling="std",
)
model = PatchTSMixerForPrediction(config)
  5. Training – Hugging Face Trainer with early stopping (patience 10) and MSE loss. Sample training log shows loss decreasing from 0.247 to ~0.12.
  6. Evaluation – Test MSE of 0.128 is reported as state‑of‑the‑art on the Electricity benchmark.
  7. Saving – Model checkpoint is stored via trainer.save_model().

Example 2 – Transfer Learning to ETTh2

The same pretrained model can be reused for a different dataset:

  • Zero‑shot – Direct evaluation on ETTh2 yields MSE 0.304, comparable to SOTA.
  • Linear probing – Freezing the backbone and training only a linear head reduces MSE to 0.271.
  • Full fine‑tuning – Unfreezing all parameters improves MSE only marginally to 0.273, indicating the pretrained features are already highly effective. All three stages use the same Trainer API with adjusted TrainingArguments (lower learning rate, early stopping patience 5).

Key Takeaways

  • PatchTSMixer is now a first‑class model in the Transformers library, simplifying integration into existing HF pipelines.
  • Its MLP‑Mixer backbone delivers strong forecasting accuracy with far lower computational overhead than traditional Transformers.
  • The model supports both direct training and transfer learning, enabling zero‑shot and fine‑tuned performance on new time‑series domains.
  • End‑to‑end notebooks cover data preparation, model configuration, training, evaluation, and checkpoint saving, providing a ready‑to‑run reference for practitioners.

Resources

Sources