xorq-labs/xorq

Git-native executable catalog for agentic data work.

What it solves

Xorq addresses the "tech-debt" accumulated by AI coding agents during data analysis. When agents generate one-off Python scripts, fragmented JSON files, and outdated requirements, the resulting work is often non-reproducible and difficult for humans or other agents to verify or productionize. Xorq replaces ephemeral prose notes and fragile scripts with an executable, content-addressed catalog of data pipelines.

How it works

Xorq turns dataframe expressions into durable, reproducible artifacts. It uses a declarative expression system (powered by Ibis) that can be compiled to multiple backends (such as DuckDB, Snowflake, or pandas).

  1. Definition: A user or agent defines a lazy dataframe expression.
  2. Build: The expression is built into a hashed directory containing a manifest (specification of the computation), a pinned Python environment (via uv), and a wheel.
  3. Cataloging: These builds are added to a catalog, which is a Git repository. This allows entries to be discovered by hash or human-readable alias, tracked via Git history, and reused by other agents or humans.

Who it’s for

  • AI Agents: Coding agents (like Claude Code) that need a way to store and retrieve executable memory of their data work to avoid starting from scratch.
  • Data Engineers: Those who want to turn ad-hoc agent-generated pipelines into portable, versioned artifacts.
  • Analysts: Users who need to reproducible data transformations that can run across different compute engines.

Highlights

  • Git-Native Storage: The catalog is a Git repo, meaning no central service is required for discovery or versioning.

  • C-addressed Pipelines: Every pipeline has a reproducible identity based on its content.

  • Multi-Engine Support: The same expression can run on embedded engines (DataFusion, DuckDB, SQLite) or warehouses (Snowflake, Databricks, Trino).

  • Agent Integration: Includes a plugin for Claude Code to automate the creation and exploration of catalogs.

  • Arrow-Native: Uses Apache Arrow for high-performance data exchange between expressions.

Related

  • Project
    dynamiq-ai/dynamiq

    An orchestration framework for agentic AI and LLM applications that streamlines the creation of RAG pipelines and multi-agent workflows.

  • Project
    johnlindquist/mdflow

    A Git-native control plane that turns Markdown files into repeatable, version-controlled AI agent workflows with built-in behavioral evaluations.

  • Dispatch
    Codex-maxxing: Transforming AI Agents into Durable Operating Loops

    Explore how to move beyond one-off prompts to create long-running, autonomous AI workflows using durable threads, shared memory vaults, and recurring heartbeats.

  • Project
    skygazer42/MimirQ

    An observable and replaceable enterprise RAG infrastructure that provides a controllable pipeline for document parsing, chunking, and retrieval with built-in regression testing.

  • Project
    conductor-oss/conductor

    An open-source durable execution platform for microservices and AI agents that turns complex workflow graphs into persisted, inspectable, and recoverable executions.