airbnb/chronon

What it solves

Chronon simplifies the complex data plumbing required for AI/ML applications, specifically for "online" models that require real-time feature serving. It eliminates the need for manual orchestration of batch and streaming pipelines and solves the problem of "training-serving skew," where the data used to train a model differs from the data it sees during production inference.

How it works

Chronon acts as an abstraction layer where users define features as transformations of raw data (using GroupBy and Join APIs). The platform then handles the rest:

• Computation: It performs batch and streaming computation to keep features up-to-date.
• Serving: It provides a low-latency API for real-time fetching of feature vectors from a KV store (e.g., MongoDB).
• Backfills: It generates point-in-time accurate historical datasets for model training and evaluation, ensuring that the backfilled data matches what would have been served online at that specific timestamp.
• Consistency Monitoring: It logs online fetch requests and compares them against backfilled values to measure and guarantee online/offline consistency.

Who it’s for

ML practitioners and engineers building real-time AI/ML models that require high-performance feature serving and guaranteed consistency between training and production data.

Highlights

• Point-in-time Correctness: Prevents label leakage by ensuring backfills are temporally accurate.
• Unified Definitions: Use a single feature definition for both offline training and online serving.
• Scalable Backfills: Handles large time windows and highly skewed data.
• Observability: Built-in tools to monitor data freshness and online/offline consistency.