oughtinc/ice

ICE – Interactive Composition Explorer

What it is – ICE is a Python library that lets you build, run, and debug recipes – structured workflows that call language‑model agents (e.g., GPT‑3/4) to solve a task. It records every LM call and shows an interactive trace in your browser, so you can see how a complex prompt chain behaved.

Key ideas

• Recipes – a hierarchy of subtasks (question‑answering, ranking, verification, …) that together solve a higher‑level problem.
• Agents – reusable building blocks that perform a single LM operation such as completion, scoring, or classification. Agents are stateless and can be swapped out (human, LM, or hybrid).
• Modes – run a recipe entirely with humans, entirely with LMs, or a mix (human+LM) to compare performance.
• Parallel execution – ICE can fire many LM calls at once, speeding up large recipes.
• Trace UI – after a run, a web UI visualises the execution graph, showing inputs, outputs, timings and any errors, which is useful for debugging and research.

Typical workflow

1. Install the package (pip install ought-ice).
2. Put your API keys in ~/.ought-ice/.env.
3. Write or import a recipe (e.g., the “Hello World” primer).
4. Call ice.run(recipe, mode="LM").
5. Open the generated trace in a browser to explore each step.

Who might use it

• Researchers experimenting with chain‑of‑thought or iterated‑decomposition prompting.
• Developers building AI assistants that need a clear, reproducible workflow.
• Anyone who wants to compare human‑in‑the‑loop vs. fully‑automated LM pipelines.

Status – Pre‑1.0; the public API can change without notice.

Links –

• Installation guide
• UI guide wiki
• Paper describing the underlying approach

---

ICE is an open‑source project from Ought, the team behind the research‑assistant tool Elicit.