superlinked/sie

Open-source inference server and production cluster for all the models your agent needs.

What it solves

SIE (Superlinked Inference Engine) eliminates the need to manage a fragmented patchwork of separate model servers for different agent tasks. It provides a single, self-hosted inference cluster that serves over 100 open-source models for various agentic needs—such as search, document processing, and content safety—through a unified API.

How it works

SIE operates as a centralized inference engine that loads models on demand and uses an LRU (Least Recently Used) eviction policy to manage memory. It provides an OpenAI-compatible API for embeddings and completions, allowing for easy migration. The system is designed for cloud deployment, shipping with Kubernetes and Helm configurations, KEDA autoscaling (including scale-to-zero), and a load-balancing gateway.

Who it’s for

It is designed for developers building AI agents who want to self-host their model infrastructure in their own cloud to maintain control over their data and reduce the complexity of managing multiple specialized model servers.

Highlights

  • Unified Model Catalog: Supports a wide array of tasks including search/reranking, document-to-markdown conversion, structured output extraction, classification, content guarding, and audio transcription.
  • OpenAI Compatibility: Offers drop-in migration via /v1/embeddings, /v1/chat/completions, and /v1/completions endpoints.
  • Production-Ready Infrastructure: Includes Helm charts, Terraform modules for major cloud providers (AWS, Azure, GCP, Alibaba), and Grafana dashboards.
  • Broad Integration: Works with popular frameworks like LangChain, LlamaIndex, Haystack, DSPy, and CrewAI, as well as vector databases like Chroma, Qdrant, and Weaviate.

Related

  • Project
    OctaFuse/octafuse-gateway

    In short: Octafuse Gateway is a self-hosted AI gateway that unifies model providers (OpenAI, Anthropic, Gemini, DashScope, MiniMax, Volcano Engine), native protocol endpoints, agent tools (search, scraping, deep search, AI detection), routing strategies, multi-tenant billing, and an admin dashboard — deployable free on Cloudflare Workers + D1 or via Docker with Postgres/MySQL.

  • Project
    swytchcodehq/swytchcode-examples

    Swytchcode Examples is a repository of production-ready AI agent workflow templates and reference architectures, showing how to connect agents to real-world APIs and tools with built-in compliance and reliability features.

  • Project
    huggingface/text-embeddings-inference

    A high-performance inference toolkit for deploying and serving open-source text embeddings and sequence classification models with low latency.

  • Project
    MattiaIppoliti/ciele

    An open-source platform for building and deploying AI assistants that answer from organizational knowledge with built-in routing, human escalation, and analytics.

  • Project
    schmitech/orbit

    A self-hosted AI backend that connects private data and tools to local or hosted models via a single OpenAI-compatible API.