SomeOddCodeGuy/WilmerAI

WilmerAI is one of the oldest LLM semantic routers. It uses multi-layer prompt routing and complex workflows to allow you to not only create practical chatbots, but to extend any kind of application that connects to an LLM via REST API. Wilmer sits between your app and your many LLM APIs, so that you can manipulate prompts as needed.

What it solves

WilmerAI provides a way to move beyond simple, keyword-based prompt routing. It solves the problem of LLMs lacking context when deciding which specialized model or tool should handle a request, and the limitation of one-shot responses that often yield lower quality results.

How it works

At its core, WilmerAI is a node-based workflow engine. It uses JSON files to define a sequence of steps (nodes) that can orchestrate multiple LLMs, call external tools, or run custom scripts.

Key mechanisms include:

  • Contextual Routing: Analyzes the entire conversation history to route prompts to the correct specialized workflow (e.g., "Coding" or "Creative").
  • Multi-LLM Orchestration: Allows different nodes within a single workflow to connect to different LLM endpoints (local or cloud), using the best model for each specific sub-task.
  • Stateful Memory: Employs a four-part memory system (chronological summaries, rolling summaries, a vector database, and a state document) to maintain long-term conversation context.
  • API Gateway: Exposes OpenAI- and Ollama-compatible endpoints, allowing it to act as a backend for existing front-end tools like Open WebUI or SillyTavern.

Who it’s for

It is designed for users who want granular control over how LLMs are orchestrated, those who run multiple local LLMs across different hardware, and developers who want to build complex, semi-autonomous agents without relying on fully autonomous, unpredictable agents.

Highlights

  • Node-Based Workflows: Complex tasks are broken down into reusable, chained-thought processes defined in JSON.
  • Advanced Routing: Uses full conversation history for both initial prompt routing and conditional "if/then" logic within workflows.
  • Distributed LLM Support: Can distribute tasks across multiple computers running different model sizes.
  • Privacy-First: Designed for local deployment with options for Fernet encryption of discussion files and request redaction in logs.
  • Tool Integration: Supports OpenAI-style tool calling and experimental agentic MCP (Model Context Protocol) server tool calling.

Related

  • Project
    NVIDIA-AI-Blueprints/llm-router

    An experimental multimodal router that analyzes user prompts to recommend the optimal LLM or VLM based on trade-offs between accuracy, speed, and cost.

  • Dispatch
    vLLM Semantic Router: Enhancing Model Performance via Micro-Agent Collaboration

    vLLM introduces the Semantic Router, a serving-layer primitive that transforms a single model API call into a bounded collaboration of micro-agents to outperform frontier models on complex benchmarks.

  • Project
    VRSEN/agency-swarm

    A framework for building multi-agent AI applications that organizes agents into collaborative swarms based on real-world organizational structures.

  • Project
    NPC-Worldwide/npcpy

    A library for building multimodal AI applications and agentic teams, featuring a unified provider interface, multi-agent debate frameworks, and a knowledge graph with sleep/dream lifecycles.

  • Project
    ulab-uiuc/LLMRouter

    An open-source library for intelligent LLM routing that dynamically selects the most suitable model for each query to optimize inference cost and performance.