reductstore/reductstore

ReductStore – Time‑indexed object storage for robotics & industrial IoT

What it is – ReductStore is an open‑source server (written in Rust) that stores arbitrary binary payloads (images, video, ROS bags, logs, sensor readings, etc.) together with a timestamp and optional labels. The data lives on a local disk that can be configured as a ring‑buffer, and the server provides a HTTP API (and a Python client) for fast ingest, time‑range + label queries, selective replication, and lifecycle policies (compression, retention).

Why it matters for AI / robotics – Edge‑AI systems constantly generate multimodal data that later needs to be replayed for debugging, model evaluation, or further analysis. Traditional stacks require a time‑series DB plus an object store plus custom scripts for retention and replication. ReductStore collapses all of that into a single service, making it easier to keep a bounded local history on robots or edge gateways and ship only the relevant subset to the cloud for training or inference pipelines.

Core capabilities

Feature	ReductStore	Typical alternatives
Arbitrary binary payloads	✅ Native records (any file type)	MinIO (objects), TimescaleDB (`BYTEA`), MongoDB (`BinData`/GridFS)
Built‑in timestamp index	✅ Every record is time‑indexed	MinIO – no time index; TimescaleDB – hypertable; MongoDB – custom schema
Label‑based queries	✅ Query by time range and label expressions	MinIO – only key listings; TimescaleDB/MongoDB – need custom tables/collections
Ring‑buffer / quota‑based retention	✅ Capacity‑based FIFO, automatic eviction	MinIO – age‑based expiration; TimescaleDB – none; MongoDB – capped collections
Selective replication	✅ Replicate only records that match label conditions	MinIO – prefix/tag only; TimescaleDB – full‑copy streaming; MongoDB – none
Lifecycle policies	✅ Automatic compression & deletion per bucket/label	MinIO – tiering on put; TimescaleDB – columnstore + retention; MongoDB – TTL only
Streaming / subscription	✅ Publish‑subscribe API (designed for storage, not a message queue)

Typical use‑cases

Robotics replay – Store camera frames, LiDAR scans, and logs with timestamps; later query “all frames from camera‑1 during the last hour when device_status=error”.
Edge‑AI data pipeline – Keep a 30‑day local buffer on a robot, compress older data, and replicate only error‑related records to a cloud ReductStore for model retraining.
Industrial IoT gateway – Collect sensor blobs from many machines, apply per‑machine labels, and enforce a 60‑day retention policy without a separate TSDB.
Selective backup – Define replication rules such as “replicate everything from bucket A where severity=high”.

Getting started (quick‑start)

# 1️⃣ Create a data directory owned by the container user (uid 10001)
mkdir -p ./data && sudo chown -R 10001:10001 ./data

# 2️⃣ Run the server (exposes HTTP API on 8383)
docker run -p 8383:8383 -v ${PWD}/data:/data \
  -e RS_API_TOKEN=my-secret-token reduct/store:latest

Or run the native binary:

curl -LO https://github.com/reductstore/reductstore/releases/latest/download/reductstore.x86_64-unknown-linux-gnu.tar.gz
tar -xzf reductstore.x86_64-unknown-linux-gnu.tar.gz
mkdir -p ./data
RS_DATA_PATH=./data RS_API_TOKEN=my-secret-token ./reductstore

Python example (async client):

import asyncio
from reduct import Client

async def main():
    async with Client("http://localhost:8383", api_token="my-secret-token") as client:
        bucket = await client.create_bucket("my-bucket", exist_ok=True)
        await bucket.write(
            "camera-1",
            b"hello, reductstore",
            timestamp="2024-01-01T10:00:00Z",
            labels={"device": "camera-1"},
        )
        async for rec in bucket.query(
            "camera-1",
            start="2024-01-01T10:00:00Z",
            stop="2024-01-01T10:00:01Z",
            when={"&device": {"$eq": "camera-1"}},
        ):
            print((await rec.read_all()).decode())

asyncio.run(main())

Full docs, tutorials, and binary downloads are at https://reduct.store.

Ecosystem & Extensibility

Client libraries – Official Python client (reduct), plus community SDKs for Go, Rust, and JavaScript.
Docker image – reduct/store (official, auto‑built on each release).
Protocol Buffers API – Server uses a vendored protoc binary; developers can generate gRPC stubs if they prefer binary RPC over HTTP.
Community – Discourse forum, GitHub issues, good‑first‑issues label, contribution guide.

When not to use ReductStore

Pure numeric time‑series data that fits a classic TSDB (e.g., Prometheus, InfluxDB).
Simple blob storage without any need for time‑based queries (plain S3 or a filesystem is cheaper).
A dedicated message‑queue requirement – ReductStore’s pub/sub is storage‑oriented, not a high‑throughput broker.

License & Activity

License – Apache‑2.0 (per repository).
Maturity – >4 years of active development, 350+ stars, 60 k+ downloads, 100+ production deployments, >1 PB of managed data.
CI / testing – GitHub Actions CI, code coverage badge, Docker pulls badge.

TL;DR

ReductStore is a purpose‑built, high‑performance storage engine for time‑indexed binary data, aimed at robotics and industrial edge AI workloads. It removes the need for a separate time‑series DB + object store by offering native timestamp indexing, label‑based queries, ring‑buffer retention, and selective replication—all accessible via a simple HTTP API and ready‑to‑run Docker image.