AIDE ML – An LLM‑driven Machine‑Learning Engineer

What it is – AIDE ML is an open‑source Python package that implements the AIDE algorithm: an LLM‑guided, tree‑search agent that writes, debugs, and benchmarks Python code for a machine‑learning task until a user‑specified performance metric is maximised (or minimised). The repo provides a thin, research‑oriented reference implementation, a CLI, a Streamlit UI, and utilities for visualising the solution‑tree.

Who it’s for

• Researchers exploring agent architectures (swap out search heuristics, evaluators, or LLM back‑ends).
• Practitioners who want an automated way to generate a high‑performing ML pipeline from a dataset and a natural‑language goal.

Key features

Feature	What you get
Natural‑language task spec	aide data_dir=… goal="Predict churn" eval="AUROC" – no YAML needed.
Agentic tree‑search	Each generated script is a node; LLM‑produced patches become children; metric feedback prunes the tree.
Model‑neutral plumbing	Works with OpenAI, Anthropic, Gemini, or any self‑hosted LLM that follows the OpenAI API (e.g., Ollama).
HTML visualiser & Streamlit UI	Inspect the full solution tree, live logs, and the best code.
Configurable steps & drafts	agent.steps, agent.search.num_drafts, temperature, token budget, reasoning effort, etc.
Docker & local‑LLM support	Run entirely on a local model (e.g., qwen2.5) via OPENAI_BASE_URL.

How to get started

1. Install the package: pip install -U aideml.
2. Export an LLM API key (OPENAI_API_KEY or ANTHROPIC_API_KEY).
3. Run the CLI with a dataset folder, a goal, and an evaluation metric. The run creates a logs/<run_id>/ directory containing the best script (best_solution.py) and an interactive tree_plot.html.

Web UI – Clone the repo, install the editable package, then streamlit run app.py inside aide/webui. Upload data, type the goal/metric, and watch the tree evolve.

Extensibility – The repo is deliberately minimal: the core Experiment class orchestrates three stages (code generation, feedback/evaluation, reporting). Each stage accepts a model ID, provider, temperature, token budget, and optional reasoning effort, making it easy to plug in new LLMs or custom evaluators.

Research impact – The algorithm is described in the arXiv paper AIDE: AI‑Driven Exploration in the Space of Code (arXiv:2502.13138). It has been cited by several high‑profile benchmarks (MLE‑Bench, RE‑Bench, AI‑Scientist‑v2, Meta’s speed‑runner, etc.), where the tree‑search version outperforms linear agents by a large margin.

Installation & testing – Besides the normal pip install, the repo ships Dockerfiles, a live end‑to‑end test (runs a few steps on the Ames housing dataset), and CI that can execute the test with real API keys.

Citation – If you use AIDE ML in a paper, cite the 2025 arXiv article provided at the end of the README.