ibis-project/ibis

What it solves

Ibis solves the problem of fragmentation in data analysis. Typically, Python dataframe libraries are tied to specific execution engines, and many databases only offer SQL interfaces. Ibis provides a single, portable Python API that works across more than 20 different backends, allowing you to write code once and run it anywhere without being locked into a specific engine.

How it works

Ibis acts as a translation layer. You write data manipulation code using its standard Python dataframe API. Ibis then compiles these "lazy" expressions into the native language of your chosen backend—usually SQL for databases or specific dataframe operations for engines like Polars or DataFusion. This allows you to switch between local development and remote deployment by changing only a single line of code.

Who it’s for

Ibis is for data scientists, analysts, and engineers who want to use the flexibility of Python for data manipulation while leveraging the power and scale of modern SQL databases or specialized execution engines. It is ideal for those who want to avoid rewriting their analysis code when moving from local prototyping to production environments.

Highlights

• Portable API: Use the same code for over 20 backends, including DuckDB, BigQuery, Snowflake, and PostgreSQL.
• Python and SQL integration: Seamlessly mix Python dataframe expressions with raw SQL queries.
• Lazy evaluation: Expressions are compiled and executed only when needed, optimizing performance.
• Interactive exploration: Designed for iterative data analysis with built-in support for local file formats like CSV and Parquet.