microsoft/pg_durable

What it solves

pg_durable provides durable execution for SQL functions, eliminating the need for developers to manually manage complex background work using a combination of cron jobs, workers, queues, and status tables. It ensures that long-running workflows survive database crashes, restarts, or individual step failures by checkpointing progress, so execution can resume from the last successful step rather than starting over.

How it works

It is implemented as a PostgreSQL extension that brings the orchestration runtime directly into the database. Users define workflows as graphs of SQL steps using a custom DSL with operators like ~> and |=>. A background worker hosts the orchestration runtime (powered by theroxide and duroxide-pg libraries), which executes these steps and persists the state of each instance to PostgreSQL tables. This allows the system to track progress, handle retries, and manage parallel execution without requiring external infrastructure like Redis or Temporal.

Who it’s for

• Backend and data engineers who want their workflow logic to live alongside their data.
• DBAs and SREs automating runbooks that must be auditable and survive system restarts.
• Teams building AI or data pipelines (e.g., vector embedding pipelines) that require durable execution per row or batch.

Highlights

• Zero Infrastructure: Runs entirely as a PostgreSQL extension with no external dependencies.
• SQL-Native: Workflows are defined and managed using a SQL-based DSL.
• Robot-like Resilience: State persists to PostgreSQL, ensuring survival through crashes and failovers.
• Database-Aware: Includes first-class primitives for scheduling, conditions, and parallel execution (fan-out/join).
• Integrated Visibility: Operational status is tracked via standard PostgreSQL tables.