superlinked/sie

Open-source inference server and production cluster for all the models your agent needs.

What it solves

SIE (Superlinked Inference Engine) eliminates the need to manage a separate model server for every different AI task. Instead of a fragmented patchwork of servers, it provides a single, self-hosted cluster that can serve over 100 different open-source models on demand, covering everything from search and retrieval to document conversion and agentic loops.

How it works

SIE acts as a unified API gateway for a wide variety of models. It uses an OpenAI-compatible API for drop-in migration and manages models using on-demand loading and LRU (Least Recently Used) eviction to optimize memory. The system is designed for production readiness, shipping with a load-balancing gateway, KEDA autoscaling (including scale-to-zero), and Terraform modules for major cloud providers (GKE, EKS, AKS).

Who it’s for

Developers building AI agents who need a centralized, self-hosted way to run multiple specialized models (e.g., for embeddings, reranking, OCR, and LLM generation) without the operational overhead of managing multiple independent inference servers.

Highlights

Unified API: Supports /v1/embeddings, /v1/chat/completions, /v1/completions, and /v1/responses for easy integration.
Broad Task Support: Handles search (embedding/reranking), document-to-markdown conversion (OCR), structured output (entity extraction), and content safety.
Production Stack: Includes Grafana dashboards, KEDA autoscaling, and cloud-specific Terraform deployments.
Extensive Integrations: Works with popular frameworks like LangChain, LlamaIndex, Haystack, DSPy, and CrewAI, as well as vector databases like Chroma and Qdrant.