aurelio-labs/semantic-router

What it solves

Semantic Router provides a high-speed decision-making layer for LLMs and AI agents. Instead of relying on slow, expensive LLM calls to determine how to handle a user's input (such as choosing which tool to use or whether to trigger a specific workflow), it performs these routing decisions in milliseconds.

How it works

It works by comparing the semantic meaning of a user's request against a predefined set of "routes" using vector space embeddings or specialized decision models. By defining Route objects—which contain example utterances—the system can classify incoming queries. It supports local execution via FastEmbed, integration with various vector databases like Pinecone and Qdrant, and calibration features that allow developers to set confidence thresholds to prevent the router from making incorrect decisions.

Who it’s for

This project is for developers building LLM applications or agents who need to add a fast, reliable control layer to manage conversation flow, tool selection, or intent classification without the latency of a full LLM generation.

Highlights

• Low Latency: Makes routing decisions in milliseconds, significantly faster than standard LLM-based dispatching.
• Flexible Routing: Supports both semantic vector-based matching and logic-based decision models.
• Calibration: Includes tools to calibrate confidence scores, allowing the system to return a "none" result if a query doesn't match any defined route.
• Extensible: Integrates with popular tools like LangChain and supports multiple storage backends including Pinecone, Qdrant, and Postgres.
• Multi-modal: Capable of routing based on image inputs in addition to text.