unitycatalog/unitycatalog

What it solves

Unity Catalog provides a universal, open-source catalog for managing data and AI assets. It solves the problem of fragmented governance by allowing users to secure and govern tabular data, unstructured assets, and AI models within a single interface, regardless of the underlying format or compute engine.

How it works

It operates as a multimodal interface with an open API (OpenAPI spec) and an OSS implementation. It supports multiple data formats (such as Delta Lake, Apache Iceberg, Apache Hudi, Parquet, JSON, and CSV) and is compatible with Apache Hive's metastore API and Apache Iceberg's REST catalog API. This allows various compute engines to read and manage the cataloged data.

Who it’s for

This project is for data engineers, AI practitioners, and organizations that need a unified way to govern and secure diverse data and AI assets across different tools and engines.

Highlights

• Universal Governance: A single interface to govern tables, files, functions, and AI models.
• Multimodal Support: Extensible support for any format (Delta Lake, Iceberg, Hudi, etc.) and any compute engine.
• Open Standard: Based on an OpenAPI specification and hosted by the LF AI & Data Foundation.
• Broad Ecosystem: Supported by a wide range of industry partners including AWS, Google Cloud, Microsoft Azure, and NVIDIA.