TangleML/tangle

What it solves

Tangle provides a visual orchestration system for machine learning experiments and production pipelines. It eliminates the complexity of manually managing batch workflows, containerized tasks, and data transfer between different steps of an ML pipeline.

How it works

Tangle uses a graph-based approach where users build pipelines by dragging and dropping components (containerized command-line programs) onto a canvas. These components are connected by linking outputs to inputs, forming a directed graph of tasks.

An API server and an Orchestrator manage the system. The Orchestrator uses "Launchers" to execute these containers on various environments—such as local Docker, Kubernetes, or Google Cloud—and handles the passing of data artifacts between tasks. It supports automatic execution caching to avoid redundant computations.

Who it’s for

It is designed for ML engineers and data scientists who need to build, monitor, and iterate on batch ML workflows that can be run locally or in the cloud.

Highlights

• Visual Pipeline Builder: An intuitive drag-and-drop interface for creating workflows.
• ** Cortana-agnostic Components**: Components can be written in any language (Python, R, Java, etc.) and are compatible with Google Cloud Vertex AI and Kubeflow Pipelines.
• Flexible Deployment: Can be run locally via Docker/Podman or hosted on HuggingFace Spaces.
• Execution Caching: Automatically re-uses successful runs from cache to save time and compute.
• Reproducibility: Maintains a full history of runs, including graphs, logs, and metadata, allowing for easy cloning and re-execution.