Output – TypeScript framework for AI workflows & agents

What it is – An open‑source TypeScript SDK that lets you define, run, and monitor end‑to‑end LLM‑powered workflows. It bundles together prompt management, evaluation, tracing, cost accounting, and orchestration (via Temporal) so you can keep the whole AI stack inside your codebase.

---

Core ideas

Idea	Why it matters
AI‑for‑AI – the framework is built so Claude Code (or similar coding agents) can read a workflow folder, generate code, and iterate automatically.	Makes “prompt‑to‑production” loops fast for teams that let LLMs write their own pipelines.
Prompt files – .prompt files with YAML front‑matter and Liquid templating are version‑controlled and provider‑agnostic.	Keeps prompts reviewable like any other source file and lets you switch models with one line.
Automatic tracing – every LLM call, HTTP request, token count, latency, and cost is written to logs/runs/.	Gives a single source of truth for debugging and for the coding agent to reason about failures.
LLM‑as‑judge evaluators – built‑in evaluators return boolean results with confidence scores and reasoning.	Enables deterministic retry loops and offline testing of model output quality.
Provider‑agnostic LLM layer – Anthropic, OpenAI, Azure, Vertex AI, Bedrock all share the same API surface.	Avoids vendor lock‑in and lets you move between models without code changes.
Temporal orchestration – workflows are deterministic coordination code; steps perform I/O and are cached for replay.	Handles retries, parallelism, child workflows, and history without you needing to manage a workflow engine yourself.
Encrypted credentials – AES‑256‑GCM encryption scoped per environment/workflow, managed via the CLI.	Prevents accidental secret leakage when sharing code or when an AI agent runs the code.

---

Main packages (npm)

Package	Role
@outputai/core	Primitives: workflow, step, evaluator definitions.
@outputai/llm	Multi‑provider LLM client, prompt loading, streaming, tool‑calling.
@outputai/cli	Project scaffolding, local dev server (Temporal + API + UI), workflow run/debug commands.

---

How a workflow looks

// src/workflows/research/workflow.ts
workflow({
  name: 'research',
  fn: async (input) => {
    const data = await gatherSources(input);
    const analysis = await analyzeContent(data);
    const quality = await checkQuality(analysis);
    return quality.passed ? analysis : await reviseContent(analysis, quality);
  }
});

workflow – deterministic orchestration, no side‑effects.

// src/workflows/research/steps.ts
step({
  name: 'gatherSources',
  fn: async (input) => {
    const results = await searchApi(input.topic);
    return { sources: results };
  }
});

step – performs I/O; result is cached for replay.

---
provider: anthropic
model: claude-sonnet-4-20250514
temperature: 0
---
<system>You are a research analyst.</system>
<user>Analyze the following sources about {{ topic }}: {{ sources }}</user>

.prompt – YAML front‑matter config + Liquid variables.

// src/workflows/research/evaluators.ts
evaluator({
  name: 'checkQuality',
  fn: async (content) => {
    const { output } = await generateText({
      prompt: 'evaluate_quality',
      variables: { content },
      output: Output.object({
        schema: z.object({
          isQuality: z.boolean(),
          confidence: z.number().describe('0-100'),
          reasoning: z.string()
        })
      })
    });
    return new EvaluationBooleanResult({
      value: output.isQuality,
      confidence: output.confidence,
      reasoning: output.reasoning
    });
  }
});

evaluator – LLM‑as‑judge that returns a structured result.

---

Getting started (quick steps)

1. Prereqs – Node 20+, Docker Desktop, an Anthropic/OpenAI/etc. API key.
2. Scaffold a new project:

   npx @outputai/cli init
   cd <project-name>
   

3. Start the local dev stack (Temporal server + UI + hot‑reload worker):

   npx output dev
   

4. Run a sample workflow, e.g. the blog evaluator:

   npx output workflow run blog_evaluator paulgraham_hwh
   

5. Inspect execution or debug with output workflow debug <id>.

Full docs are in docs/guides/start-here/getting-started.mdx.

---

Example workflow gallery

The companion repo output‑examples ships ready‑to‑run workflows such as:

• blog_evaluator – quality scoring of blog posts (uses Jina Reader).
• call_scorer – rate sales‑call transcripts against MEDDIC/BANT/SPIN.
• changelog_generator – turn GitHub commits/PRs into a categorized changelog.
• dependency_audit – scan npm packages for vulnerabilities via OSV.
• recipe_extractor, url_summarizer, youtube_summarizer – content extraction & summarisation.
• ai_hn_digest – personalized Hacker News newsletter published to Beehiiv.
• sales_call_processor – multi‑step processing of call transcripts.

All can be cloned, edited, and run locally to see the framework in action.

---

Who’s using it?

• CheckThat.ai – an AI‑enhanced brand‑monitoring platform that runs deterministic LLM workflows built on Output, tracking millions of AI‑generated brand mentions.

---

When to choose Output

• You want one code‑first stack for prompts, tracing, evaluation, and orchestration.
• Your team (or an LLM‑coding agent) needs to generate or modify workflows automatically.
• You need vendor‑agnostic LLM calls and a way to switch providers without code churn.
• You require reliable retries, parallelism, and replay – Temporal handles that under the hood.
• You prefer self‑hosted secrets management rather than a separate vault service.

---

License

Apache 2.0 – free for commercial and open‑source use.

---

Bottom line – Output is a full‑stack, TypeScript‑first framework that unifies the disparate pieces of an LLM application (prompts, runs, costs, evaluations) and adds production‑grade workflow orchestration, making it easier for both human engineers and AI coding agents to build, test, and ship AI‑driven products.