microsoft/pg_durable

What it solves

pg_durable provides a way to run long-running, fault-tolerant workflows directly inside PostgreSQL. It eliminates the need for external orchestrators (like Airflow or Temporal) or complex manual setups involving cron jobs, worker queues, and status tables to manage background tasks that must survive database crashes or restarts.

How it works

It is a PostgreSQL extension that allows users to define workflows as graphs of SQL steps using a specific DSL (with operators like ~> and |=>). A background worker hosts a runtime that executes these steps and checkpoints progress. If a failure occurs, the system resumes execution from the last durable checkpoint rather than restarting the entire process.

Who it’s for

• Backend and data engineers who want their workflow logic to live alongside their data.
• DBAs and SREs automating runbooks that need to be auditable and resilient to restarts.
• Teams building AI or data pipelines (e.g., vector embedding pipelines) that need durable execution per row or batch.

Highlights

• Zero Infrastructure: Runs entirely as a PostgreSQL extension with no need for Redis or external services.
• SQL-Native: Workflows are defined and managed using SQL and composable operators.
• Fault-Tolerant: State persists to PostgreSQL, allowing workflows to survive crashes, restarts, and failovers.
• Database-Aware: Includes built-in primitives for scheduling, parallel execution, and conditional branching.