openlake: a high-throughput distributed object store for GPU workloads that bypasses the host CPU and kernel

What it solves

OpenLake is designed to eliminate the bottlenecks in AI infrastructure by reducing the time spent moving data from storage to GPU memory. It addresses the overhead caused by host CPUs, page caches, and multiple userspace copies that typically slow down training and inference clusters.

How it works

OpenLake is a distributed object store built with Rust and io_uring using a thread-per-core architecture. It employs several high-performance techniques:

• Zero-copy data movement: Uses GPUDirect Storage and RDMA to move data directly from the network interface card (NIC) into GPU VRAM, bypassing the host memory and kernel.
• Bypassing the kernel: The HTTP frontend and storage engine run on the same thread to prevent requests from crossing core boundaries.
• Efficient storage: Uses SIMD Reed Solomon erasure coding to reduce storage costs while maintaining high throughput.
• Congestion control: Implements PacedRDMA, a credit-based system to handle request bursts and minimize tail latencies.

Who it’s for

It is intended for developers and engineers building AI infrastructure, specifically those managing high-throughput GPU workloads for training and inference.

Highlights

• High Performance: Delivers over a million IOPS with sub-1ms latency.
• S3 Compatibility: Can be interacted with using any standard S3 client.
• Rust-based: Built for safety and memory efficiency using the Rust language.
• Hardware Acceleration: Leverages GPUDirect Storage and RDMA for maximum throughput.