Weave Router: Intelligent Model Routing for Agentic Systems

Optimizing LLM Costs with Intelligent Routing

Weave Router is a drop-in proxy for Anthropic, OpenAI, and Gemini that automatically selects the most cost-effective model for every individual request. By replacing a standard API endpoint with the Weave Router endpoint, users can reduce LLM costs by 40-70% while maintaining performance, with routing decisions made in under 50ms.

Unlike "vibes-based" prompting for routing, Weave Router utilizes a tiny on-box embedder and a cluster scorer derived from the Avengers-Pro research (Beyond GPT-5: Making LLMs Cheaper and Better via Performance–Efficiency Optimized Routing). It is currently ranked #1 on the RouterArena leaderboard with an Acc-Cost Arena score of 76.09.

Core Features and Capabilities

Multi-Provider Support

Weave Router acts as a unified interface for several major AI providers and open-source models:

  • Native APIs: Full support for Anthropic Messages, OpenAI Chat Completions, and Gemini native APIs, including streaming, tool use, and vision.
  • OSS Integration: Access to DeepSeek, Kimi, GLM, Qwen, Llama, and Mistral via OpenRouter or any OpenAI-compatible endpoint.
  • Security: Uses a "Bring Your Own Key" (BYOK) model where provider keys are encrypted at rest and stored locally on the user's machine.

Observability and Integration

The router includes built-in observability via OTLP traces, allowing users to monitor routing decisions and costs through the Weave dashboard or external tools like Honeycomb, Datadog, and Grafana.

Integration with Developer Tools

Weave Router is designed to be integrated into existing agentic workflows and IDEs with minimal configuration:

  • Claude Code: Can be wired via make install-cc or the npx @workweave/router command.
  • Codex (OpenAI CLI): Patches the configuration to use the Weave provider while allowing existing OpenAI keys to flow through.
  • opencode: Merges a provider entry into the configuration, utilizing the router's native Anthropic Messages API support.
  • Cursor: Supports integration via the "Override OpenAI Base URL" setting in the Models menu.

Users can toggle the router on or off for specific clients using commands like npx @workweave/router off --claude or via slash commands within Claude Code (/router-off, /router-on).

Technical Implementation and Endpoints

Deployment Options

Users can deploy the router in two ways:

  1. Hosted: Using npx @workweave/router, which handles configuration and wiring for specific tools.
  2. Self-Hosted: Using make full-setup to boot a Postgres database and the router on localhost:8080.

API Endpoints

Endpoint Format Function
POST /v1/messages Anthropic Messages Routed request
POST /v1/chat/completions OpenAI Chat Completions Routed request
POST /v1beta/models/:action Gemini generateContent Routed request
POST /v1/route Custom Returns routing decision without calling upstream
GET /v1/models Anthropic Passthrough

Community Insights and Technical Trade-offs

Discussion among technical users highlights several critical considerations when implementing proxy-level routing in agentic systems:

The Prompt Caching Conflict

Multiple contributors pointed out that frequent model switching can invalidate prompt caches, potentially increasing costs and latency.

"Caching and optimum routing of a single request are at odds with each other. Higher the distinct model count in a conversation, more cache misses you accept."

Agent-Aware Routing

Some users argued that sophisticated coding agents already perform their own routing (e.g., routing discovery to "mini" models and planning to "heavy" models). A proxy-level router may break this internal control loop if the agent is unaware that the proxy has changed the model.

Model-Specific Prompting

There is a concern that prompts are often tuned for specific models. If a router changes the model mid-conversation based on the prompt's diction, it may lead to suboptimal results if the prompt was specifically engineered for a different model's behavior.

Performance Evaluation

Users have requested more transparent benchmarks, specifically running the router against industry-standard evaluations like TerminalBench or DeepSwe Bench to prove the cost-vs-performance gains in real-world agentic coding tasks.

Sources

Related

  • Project
    NVIDIA-AI-Blueprints/llm-router

    An experimental multimodal router that analyzes user prompts to recommend the optimal LLM or VLM based on trade-offs between accuracy, speed, and cost.

  • Project
    workweave/router

    A drop-in LLM proxy that automatically routes every request to the best-performing model using a local embedder and cluster scoring.

  • Dispatch
    vLLM Semantic Router: Enhancing Model Performance via Micro-Agent Collaboration

    vLLM introduces the Semantic Router, a serving-layer primitive that transforms a single model API call into a bounded collaboration of micro-agents to outperform frontier models on complex benchmarks.

  • Project
    ulab-uiuc/LLMRouter

    An open-source library for intelligent LLM routing that dynamically selects the optimal model for each query to balance cost and performance.

  • Project
    NadirRouter/NadirClaw

    A local LLM routing proxy that cuts API costs by 40-70% by routing prompts to the cheapest viable model and escalating to premium models only when necessary.