lakekeeper/lakekeeper

Apache Iceberg REST Catalog in Rust — access control, credential vending and audit for every engine and AI agent. Apache 2.0.

What it solves

Lakekeeper provides a secure, centralized way to manage Apache Iceberg tables across different compute engines. It eliminates the need to duplicate access control rules for every engine (like Spark, Trino, or Starrocks), ensuring that security policies are defined once in the catalog and enforced consistently across the entire data lakehouse.

How it works

It is a Rust-based implementation of the Apache Iceberg REST Catalog specification. Lakekeeper acts as a gateway that checks every request against a defined policy before data is read. It manages storage access using vended credentials and remote signing for major cloud providers (AWS, Azure, GCP) and on-premise S3 stores. It integrates with OpenID providers for authentication and uses OpenFGA by default for fine-grained access control.

Who it’s for

Data engineers and platform architects building open lakehouses who need a unified, scalable, and secure catalog to manage table metadata and access permissions across multiple query engines.

Highlights

  • Unified Access Control: Define security policies once and enforce them across all connected compute engines.
  • Rust-powered: Delivered as a single binary without requiring a JVM or Python environment.
  • Storage Agnostic: Supports S3, Azure ADLS Gen2, Google Cloud Storage, and Microsoft OneLake.
  • Extensible Architecture: Provides traits for customizing the database implementation, secret stores, authorizers, and event backends.
  • Change Tracking: Built-in support for emitting CloudEvents to react to table changes.
  • Multi-tenant: A single deployment can serve multiple projects and warehouses.

Related

  • Project
    lakesoul-io/LakeSoul

    LakeSoul is an open‑source lakehouse framework with a Rust‑native metadata and I/O core, offering ACID transactions, incremental upserts, automatic compaction, fine‑grained RBAC, and support for Spark, Flink, Presto, Ray, Daft, and Python‑based tools. It lets you store regular rows and multimodal/vector data (via the Vortex format) and provides real‑time CDC ingestion, snapshot queries, and seamless integration with AI/ML pipelines.

  • Project
    unitycatalog/unitycatalog

    An open-source, universal catalog for the unified governance and security of multimodal data and AI assets across various formats and compute engines.

  • Project
    rustfs/rustfs

    A high-performance, distributed object storage system built in Rust that is S3-compatible and optimized for AI and big data workloads.

  • Project
    databricks/databricks-sdk-py

    A Python client library that wraps all Databricks REST APIs, handling authentication (PAT, Azure, GCP, OIDC), retries, pagination, and long‑running operations, enabling programmatic control of clusters, jobs, Unity Catalog, and other Lakehouse resources.

  • Project
    databendlabs/databend

    An open-source enterprise data warehouse built in Rust that integrates analytics, vector search, and secure Python sandboxes for AI agent orchestration.