airbytehq/PyAirbyte

PyAirbyte brings the power of Airbyte to every Python developer. Powers the Airbyte Cloud Replication MCP.

What it solves

PyAirbyte allows Python developers to use Airbyte connectors directly within their Python code, removing the need for the full Airbyte platform for simple data movement tasks. It provides a lightweight way to ingest data from various sources and move it to a destination or a local cache.

How it works

PyAirbyte provides utilities to install and run Airbyte connectors. It supports three installation methods: declarative sources (YAML-based), Python-based installations (using uv or pip), and Docker images. Users can specify Python versions for specific connectors to ensure compatibility.

Who it’s for

Python developers who want to leverage Airbyte's extensive library of connectors without deploying the full Airbyte orchestration platform, as well as developers building ETL pipelines or testing Airbyte sources.

Highlights

  • Flexible Installation: Supports YAML, Python (via uv or pip), and Docker for connector deployment.
  • Fast Connector Setup: Uses uv by default for faster Python connector installation.
  • Custom Python Versions: Allows specifying a specific Python version for each connector to avoid version conflicts.
  • Cachable Data: Includes a built-in cache implementation for landing data.
  • Orchestration Compatible: Works with data orchestration frameworks like Airflow, Dagster, and Snowpark.

Related

  • Project
    airbytehq/airbyte

    Airbyte is an open-source data movement platform that syncs data from hundreds of sources to warehouses or AI agents, providing an SDK to turn these connectors into LLM-ready tools.

  • Project
    Kaggle/kagglehub

    A Python library for programmatically downloading, uploading, and loading Kaggle datasets, models, and competition resources into data science frameworks.

  • Project
    fal-ai/fal

    A serverless Python runtime and client library for deploying and scaling ML models and pipelines in the cloud without managing infrastructure.

  • Project
    huggingface/kernels

    A system for dynamically loading and distributing optimized compute kernels from the Hugging Face Hub, enabling portable and version-controlled hardware acceleration.

  • Project
    flyteorg/flytekit

    Flytekit Python is the official SDK for Flyte, letting you write, test, and deploy AI/ML workflows in pure Python using `@task` and `@workflow` decorators. Install with `pip install flytekit`, follow the quick‑start guide, and extend via plugins or contribute through the provided docs.