pathwaycom/pathway-benchmarks

What it solves

This repository provides a standardized way to measure and compare the performance of the Pathway Live Data Framework against other industry-standard streaming and batch data processing engines like Apache Flink, Apache Spark, and Kafka Streams. It specifically addresses the need for reproducible benchmarks for high-throughput, low-latency real-time data processing.

How it works

The project implements two primary benchmark suites:

• WordCount Benchmark: Measures online streaming performance by reading words from a Kafka stream and counting them, reporting 95% latency relative to throughput.
• PageRank Benchmark: Evaluates iterative graph processing across three modes: batch, streaming, and "backfilling" (a hybrid mode that switches from batch to online processing mid-way).

These benchmarks are executed within Docker containers to ensure consistent resource limits (CPU and RAM) across different frameworks, using a unified Python API for Pathway.

Who it’s for

• Data engineers and architects evaluating streaming data frameworks.
• Performance researchers comparing the efficiency of Rust-based engines versus JVM-based engines (like Flink and Spark).
• Users of the Pathway Live Data Framework who want to verify performance claims on their own hardware.

Highlights

• Comparative Analysis: Directly compares Pathway against Flink, Spark, and Kafka Streams.
• Hybrid Workloads: Specifically tests "backfilling" capabilities, measuring the ability to transition from batch to streaming processing.
• Reproducible Environment: Provides Dockerized scripts and dataset access (including the Stanford LiveJournal dataset) to ensure results can be replicated.