aurelio-labs/semantic-router

Superfast AI decision making and intelligent processing of multi-modal data.

What it solves

Semantic Router 提供一个高速决策层给 LLM 与 AI 代理。它消除必须等待慢速的 LLM 生成来决定使用哪个工具或如何路由请求的需求，降低延迟并提升响应速度。

How it works

此项目不使用 LLM 来分类查询，而是使用语义向量空间。开发者可以定义包含一组示例语句的 Route 对象。当用户查询到达时，项目会将查询编码为向量，并与路由的语句向量比较，以语义相似度决定最合适的路径。

Who it’s for

构建 LLM 驱动应用和 AI 代理的开发者，需要快速且高效的意图分类和请求路由，而不依赖昂贵或慢速的 LLM 调用。

Highlights

  • Fast Decision Making: Uses vector embeddings rather than LLM generation for routing decisions.
  • Flexible Encoders: Supports multiple embedding providers including Cohere, OpenAI, Hugging Face, and FastEmbed.
  • Local Execution: Ability to run fully local versions using HuggingFaceEncoder and LlamaCppLLM.
  • Vector DB Integration: Integrates with Pinecone and Qdrant for managing utterance vector spaces.
  • Multi-modal Support: Capable of routing based on multi-modal inputs (e.g., identifying images).
  • Agent Framework Integration: Works with LangChain Agents.