aurelio-labs/semantic-router

Superfast AI decision making and intelligent processing of multi-modal data.

What it solves

Semantic Router 提供一個高速決策層給 LLM 與 AI 代理。它消除必須等待緩慢的 LLM 產生來決定使用哪個工具或如何路由請求的需求，降低延遲並提升回應速度。

How it works

此專案不使用 LLM 來分類查詢，而是使用語意向量空間。開發者可以定義包含一組範例語句的 Route 物件。當使用者查詢到達時，專案會將查詢編碼成向量，並與路由的語句向量比較，以語意相似度決定最適合的路徑。

Who it’s for

建構 LLM 驅動應用與 AI 代理的開發者，需快速且有效率的意圖分類與請求路由，且不想依賴昂貴或緩慢的 LLM 呼叫。

Highlights

  • Fast Decision Making: Uses vector embeddings rather than LLM generation for routing decisions.
  • Flexible Encoders: Supports multiple embedding providers including Cohere, OpenAI, Hugging Face, and FastEmbed.
  • Local Execution: Ability to run fully local versions using HuggingFaceEncoder and LlamaCppLLM.
  • Vector DB Integration: Integrates with Pinecone and Qdrant for managing utterance vector spaces.
  • Multi-modal Support: Capable of routing based on multi-modal inputs (e.g., identifying images).
  • Agent Framework Integration: Works with LangChain Agents.