World Model Optimizer: Distill and Serve Frontier Models at Half the Cost

World Model Optimizer: Distill and Serve Frontier Models at Half the Cost

World Model Optimizer enables continuous model improvement using agent traces

The tool turns existing agent traces into a loop of world‑model simulation, meta‑harness optimization, and model distillation to produce a model that matches frontier quality while reducing cost.

Getting started with wmo

First install the package, register model providers, and build a router from your OTel traces.

  • Install: pip install world-model-optimizer
  • Register providers: wmo providers set
  • Build a router: wmo build --file traces.jsonl --name my-endpoint
  • Score models on held‑out tasks: wmo optimize route sweep my-endpoint --traces traces.otel.jsonl
  • Fit a routing policy: wmo optimize route fit matrix.json --kind knn --out .wmo/models/my-endpoint/policy.json
  • Serve the endpoint: wmo serve --name my-endpoint

Evaluating cost and quality gains

After serving, you can compare the new endpoint against a baseline model (e.g., gpt‑5.5) using the report command. wmo optimize route report matrix.json .wmo/models/my-endpoint/policy.json --baseline gpt-5.5 This produces metrics that show the improvement delta and cost reduction.

Distilling a small model into the pool

To avoid routing and serve a single optimized model, use the model distillation subcommand. wmo optimize model (see the distill README for details) creates a small model that inherits the performance of the routed pool. You can then pin a model with wmo optimize route pin or build an optimized harness via wmo optimize harness.

Hosted platform usage

Creating an account at platform.experientiallabs.ai lets you run agents without managing credentials. After logging in:

  • wmo login
  • Run an agent’s champion harness: wmo run <agent-id> The platform handles model and sandbox authentication, so local API keys are not required.

Using the E2B backend for isolated evaluation

If you want to evaluate optimizations in isolated sandboxes, install the E2B extra and provide an API key. pip install "world-model-optimizer[e2b]" export E2B_API_KEY=... Then run harness optimization or evaluation inside E2B: wmo optimize harness my-agent my-environment --tasks tasks.jsonl --backend e2b wmo eval tasks.jsonl --mode closed-loop --harness my-agent --harness-backend e2b Every candidate is measured against the same simulated tasks, and only changes that pass evaluation gates become the new versioned champion harness.

World model as an API

The package includes world models that can be queried programmatically or over HTTP. Python example:

from wmo import Action, ActionKind
from wmo.config.store import WorldModelStore
from wmo.engine.loader import load_world_model

model_dir = WorldModelStore(\"\.wmo\\)).resolve(\"airline\
wm, _provider = load_world_model(model_dir)
session = wm.new_session(task=\"check out the cart\
obs = wm.step(session.id, Action(kind=ActionKind.TOOL_CALL, name=\"add_to_cart\", arguments={\"sku\": \"A1\"}))
print(obs.content)

HTTP endpoints: GET /world_models lists models, then POST /world_models/{name}/sessions creates a session, and POST /world_models/{name}/sessions/{id}/step advances it.

Workspace upload and detach/reattach workflow

After login, you can run agents with live‑sync of local changes.

  • Attach workspace: wmo run <agent-id> -u . --task \"fix the failing tests\"
  • Detach to continue in the platform: wmo run <agent-id> -u . --detach
  • Send a message to the detached agent: wmo run --send \"Now run the full test suite\"
  • Reattach later: wmo run --attach
  • End the run: wmo run --end

Development setup

The project is managed with uv, using ruff for linting/formatting and ty for type checking.

  • Install dev environment: uv sync --extra dev
  • Lint: uv run ruff check .
  • Format: uv run ruff format .
  • Type check: uv run ty check
  • Run tests: uv run pytest -q

Usage telemetry

wmo collects anonymous metadata only; no prompts, traces, actions, observations, file paths, model names, or credentials are stored. Telemetry is enabled by default. To disable for a project: uv run wmo config telemetry disable This writes .wmo/settings.toml. Re‑enable with uv run wmo config telemetry enable, check status with uv run wmo config telemetry status, or disable per‑process via DO_NOT_TRACK=1 or WMO_TELEMETRY=0.

Community feedback on Hacker News

Comments highlighted both enthusiasm and skepticism.

  • One user asked for concrete proof: "The absolute best way to prove this works is by releasing a model that was fine‑tuned with this method and then showing benchmarks depicting the improvement delta between the base model and the fine tuned one. … Until then, this is noise." — @Art9681
  • Another noted the focus on routing: "The title is misleading. This is model routing, not distillation." — @surround
  • A commenter questioned cost calculation for local models: "Not sure I get it. The model you’re improving is local? If so how do you even calculate cost compared to an API" — @jack_pp
  • Positive remarks included: "Cool project" — @digitaltrees, "Cool idea! How do you guarantee privacy?" — @yiyingzhang, and "Excited to play with this." — @rglover

These reactions show interest in benchmarking, clarification on routing versus distillation, privacy guarantees, and cost modeling for local versus API models.

Sources