OpenTSLM/OpenTSLM

OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data

OpenTSLM – Time‑Series Language Models for Medical Data

What it is – OpenTSLM is an open‑source library that extends large language models (LLMs) so they can ingest and reason over multivariate time‑series data (e.g., ECG, accelerometer, EEG). The project provides:

  • Pre‑trained “TSLM” checkpoints built on Llama 3.2 (1 B/3 B) and Gemma (270 M/1 B) that understand any length of time‑series and can be prompted in natural language.
  • A Python API (OpenTSLM class) that loads these checkpoints from the Hugging‑Face Hub and offers a generate method for inference.
  • Curriculum‑style training scripts that let researchers fine‑tune the models on a sequence of tasks (multiple‑choice QA → captioning → chain‑of‑thought reasoning) covering human‑activity recognition, sleep staging, ECG QA, and generic time‑series captioning.
  • Demo scripts for quick end‑to‑end runs on each benchmark dataset.

Quick start (inference)

pip install opentslm          # install the library

from opentslm import OpenTSLM
from opentslm.time_series_datasets.TSQADataset import TSQADataset
from opentslm.time_series_datasets.util import extend_time_series_to_match_patch_size_and_aggregate
from torch.utils.data import DataLoader
from opentslm.model_config import PATCH_SIZE
import torch

REPO_ID = "OpenTSLM/llama-3.2-1b-tsqa-sp"
model = OpenTSLM.load_pretrained(REPO_ID,
                                 device="cuda" if torch.cuda.is_available() else "cpu")

test_dataset = TSQADataset("test", EOS_TOKEN=model.get_eos_token())
loader = DataLoader(test_dataset,
                    batch_size=1,
                    shuffle=False,
                    collate_fn=lambda b: extend_time_series_to_match_patch_size_and_aggregate(
                        b, patch_size=PATCH_SIZE))

for batch in loader:
    preds = model.generate(batch, max_new_tokens=200)
    for sample, out in zip(batch, preds):
        print("Q:", sample.get("pre_prompt", "N/A"))
        print("A:", sample.get("answer", "N/A"))
        print("Model output:", out)
    break   # demo shows first few examples

The script loads a soft‑prompt‑tuned model (sp) and runs a few inference steps on the TSQA test set.

Training (curriculum learning)

OpenTSLM ships a curriculum_learning.py driver that runs five staged training phases:

  1. Stage 1 – MCQ (multiple‑choice QA on the TSQA dataset)
  2. Stage 2 – Captioning (free‑form description of time‑series, M4 dataset)
  3. Stage 3 – CoT (chain‑of‑thought reasoning for Human Activity Recognition)
  4. Stage 4 – Sleep CoT (sleep‑stage classification)
  5. Stage 5 – ECG CoT (ECG question answering)

Typical command:

python curriculum_learning.py \
    --model OpenTSLMFlamingo \
    --llm_id meta-llama/Llama-3.2-1B \
    --device cuda \
    --stages stage1_mcq stage2_captioning stage3_cot

The script automatically loads the best checkpoint from the previous stage, saves new checkpoints under results/<llm_id>/<model_type>/stageX/, and writes metrics and predictions to JSONL files.

Model variants

Base LLM Size Variant Repo ID pattern
Llama‑3.2 1 B Soft‑Prompt (sp) OpenTSLM/llama-3.2-1b-<dataset>-sp
Llama‑3.2 3 B Flamingo (flamingo) OpenTSLM/llama-3.2-3b-<dataset>-flamingo
Gemma 270 M sp OpenTSLM/gemma-3-270m-<dataset>-sp
Gemma 1 B flamingo OpenTSLM/gemma-3-1b-pt-<dataset>-flamingo

The library works with any of the listed models; you only need a Hugging‑Face token with read access to the Meta or Google model repositories.

Key components

  • opentslm/ – core library (model wrappers, dataset utilities, config constants).
  • demo/huggingface/ – ready‑to‑run scripts for each benchmark.
  • scripts/ – memory‑usage analysis, dataset creation, and auxiliary tooling.
  • curriculum_learning.py – orchestrates multi‑stage training/evaluation.
  • results/ – default output layout for checkpoints and metrics.

License & community

  • License: MIT (REUSE‑compatible).
  • Contributing: Guidelines and a code‑of‑conduct are provided; contributions are welcomed via pull‑requests.
  • Contact / research opportunities: email digitalhealthresearch@stanford.edu or visit the Stanford/ETH labs’ student‑research page.

When to use OpenTSLM

  • You need a single model that can answer natural‑language questions about clinical time‑series (e.g., “What arrhythmia is present in this ECG?”).
  • You want to fine‑tune a language model on a new medical time‑series dataset while preserving the LLM’s reasoning abilities.
  • You are researching multimodal LLMs and need a reference implementation that treats time‑series as a first‑class modality.

Bottom line: OpenTSLM turns off‑the‑shelf Llama or Gemma models into time‑series language models that can be prompted like any LLM, while still handling arbitrarily long multivariate signals. The repository supplies pre‑trained checkpoints, a clean Python API, and a curriculum‑training pipeline for extending the approach to new medical domains.

Related

  • Project
    zai-org/GLM-5

    GLM‑5 is an open‑source series of very large LLMs (320 B‑744 B total parameters) optimized for multimodal input, 1 M‑token contexts, and long‑horizon agentic tasks. It introduces hybrid sparse/linear attention, IndexShare, and a custom RL engine (slime) to achieve frontier‑level performance on coding and planning benchmarks while offering cheap FP8 inference. Models are downloadable from Hugging Face/ModelScope and can be served with SGLang, vLLM, KTransformers, etc., with optional ‘reasoning_effort’ control.

  • Project
    alexzhang13/rlm

    An inference engine and training environment for Recursive Language Models (RLMs) that enable LLMs to handle near-infinite context by programmatically decomposing tasks via recursive sub-calls in a REPL environment.

  • Dispatch
    GLM-5.2: Built for Long-Horizon Tasks

    GLM-5.2, released by Z.AI on Hugging Face, introduces a solid 1M-token context, improved architecture via IndexShare and MTP enhancements, effort-level control, and strong open-source performance on long-horizon coding benchmarks.

  • Dispatch
    Open FinLLM Leaderboard launch – comprehensive zero‑shot benchmark for financial language models

    Hugging Face launched the Open FinLLM Leaderboard, a zero‑shot benchmark covering 40 finance‑specific tasks across seven categories to evaluate LLM readiness for real‑world financial applications.

  • Project
    meta-pytorch/torchtune

    torchtune is a PyTorch‑based library that provides ready‑to‑run recipes and configs for fine‑tuning, RL‑HF, distillation, quantization‑aware training, and inference of many modern LLMs (Llama, Gemma, Qwen, Phi, etc.). It abstracts distributed training, memory‑saving tricks, and device support behind a simple CLI (`tune`) and YAML files, integrating with Hugging Face, torch‑ao, FSDP2, and logging tools. Though development stopped in 2025, it remains a practical, open‑source toolkit for LLM experimentation.