polyaxon/haupt

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

Haupt – Polyaxon’s auxiliary service

What it is

  • Haupt is a component of the Polyaxon platform, a suite of tools for managing machine‑learning workflows.
  • It runs as a service that exposes several APIs and interactive utilities for working with experiments and their outputs.

Core capabilities (as listed in the README)

Feature What it does
Lineage metadata API Provides programmatic access to the provenance information of runs (e.g., which data, code, and parameters produced a given result).
Artifacts / metrics / series streams Lets you push and retrieve files, scalar metrics, and time‑series data generated during training or evaluation.
Sandbox, interactive sessions, SSH, tmux Offers a lightweight, isolated environment where users can open a shell, attach via SSH, or use tmux for persistent terminal sessions.
API for local viewer Supplies an endpoint that front‑ends can call to render results (plots, logs, etc.) locally.
Spaces and notebook services Hosts collaborative spaces and Jupyter‑style notebooks that can be launched directly from the platform.

Why it matters

  • In modern ML pipelines, keeping track of what produced a model (data versions, code commits, hyper‑parameters) is essential for reproducibility; Haupt’s lineage API helps automate that.
  • Streaming artifacts and metrics in real time lets engineers monitor training jobs without leaving the platform.
  • The interactive sandbox/SSH/tmux support means developers can debug or explore runs in a familiar terminal environment, reducing context‑switching.
  • By exposing a local‑viewer API and notebook services, Haupt bridges the gap between raw experiment outputs and human‑readable dashboards or exploratory notebooks.

Who should use it

  • Teams that already use Polyaxon for experiment tracking and want a built‑in way to serve metadata, artifacts, and interactive sessions.
  • Researchers needing reproducible pipelines with clear lineage information.
  • DevOps or MLOps engineers looking to integrate ML run data into broader monitoring or CI/CD systems.

Where to learn more

Related

  • Project
    polyaxon/polyaxon

    Polyaxon is a platform for building, training, and monitoring large-scale deep learning applications, solving problems related to reproducibility, automation, and scalability.

  • Project
    datahub-project/datahub

    An open-source AI data catalog and metadata platform that enables data discovery, governance, and observability across fragmented data ecosystems.

  • Project
    hatchet-dev/hatchet

    Hatchet is an open‑source orchestration engine that lets you run reliable background tasks, AI‑agent pipelines and durable workflows. It stores all state in PostgreSQL, provides retries, scheduling, DAGs, rate‑limiting, priority queues and a real‑time UI, and offers SDKs for Python, TypeScript, Go and Ruby. You can use it as a managed cloud service or self‑host it via Docker.

  • Project
    joeseesun/qiaomu-meta-skill

    A meta-skill that transforms prompts and workflows into researched, evaluated, and securely published AI agent skill packages.

  • Dispatch
    Codex-maxxing: Transforming AI Agents into Durable Operating Loops

    Explore how to move beyond one-off prompts to create long-running, autonomous AI workflows using durable threads, shared memory vaults, and recurring heartbeats.