databricks/databricks-sdk-py

What it solves

This SDK provides a programmatic way for developers to interact with the Databricks Lakehouse platform. It eliminates the need to manually write HTTP requests to the Databricks REST API by providing a robust, typed Python interface for all public API operations.

How it works

The library centers around the WorkspaceClient and AccountClient classes, which handle the connection to Databricks workspaces and accounts. It features an internal HTTP client that manages intelligent retries and handles paginated responses and long-running operations automatically. The SDK supports a flexible authentication flow that can automatically detect credentials from environment variables, configuration profiles (.databrickscfg), or cloud-native identity providers (AWS, Azure, and GCP).

Who it’s for

Python developers and data engineers who need to automate the management of Databricks clusters, jobs, and Unity Catalog resources, or integrate Databricks functionality into external applications.

Highlights

• Full API Coverage: Covers all public Databricks REST API operations.
• Automated Auth: Supports a default authentication flow that auto-detects credentials across different cloud providers.
• Robustness: Built-in intelligent retries for handling API failures.
• Cloud Native: Deep integration with Azure and GCP native authentication methods.
• Developer Friendly: Provides autocompletion support and integrated examples for common use cases like OAuth webservers.