superlinked/sie

Open-source inference server and production cluster for all the models your agent needs.

What it solves

SIE (Superlinked Inference Engine) eliminates the need to manage a separate model server for every individual task in an AI agent's workflow. Instead of a patchwork of different servers for embedding, OCR, and text generation, it provides a single, self-hosted cluster that serves over 100 open-source models on demand.

How it works

SIE acts as a unified API gateway that loads models from Hugging Face as needed and evicts them using a Least Recently Used (LRU) strategy to manage memory. It provides an OpenAI-compatible API for easy migration and supports multiple modalities through specialized Docker images (e.g., for OCR or GPU-accelerated generation). For production, it includes Kubernetes and Helm configurations for load balancing, KEDA autoscaling, and monitoring via Grafana.

Who it’s for

Developers building AI agents who want to host their own open-source models in their own cloud without the operational overhead of managing multiple disparate inference servers.

Highlights

  • Unified Model Catalog: Supports a wide range of tasks including search/retrieval (BGE-M3, ColBERT), document-to-markdown (Docling, PaddleOCR), structured output (GLiNER), and agent loops (Qwen).
  • OpenAI-Compatible: Drop-in replacement for standard API endpoints like /v1/chat/completions and /v1/embeddings.
  • Production-Ready Infra: Ships with Helm charts, KEDA autoscaling (scale-to-zero), and Terraform modules for major cloud providers (AWS, GCP, Azure, Alibaba).
  • ** لقب Broad Integration**: Works with popular frameworks like LangChain, LlamaIndex, and DSPy, as well as vector databases like Chroma and Qdrant.

Related

  • Dispatch
    Cohere Integration with Hugging Face Inference Providers

    Hugging Face has added Cohere as a supported Inference Provider, allowing users to run serverless inference for a wide range of Cohere and Cohere Labs models directly on the Hub.

  • Project
    Tracer-Cloud/opensre

    An open-source framework for building AI SRE agents that automate production incident response by correlating data across logs, metrics, and traces.

  • Project
    CyberStrikeus/CyberStrike

    CyberStrike is an open‑source AI agent that turns any LLM (Claude, GPT, Gemini, etc.) into an autonomous penetration‑testing platform. It adds a security‑focused “intelligence layer”, ships 13+ specialised agents and 7,600+ signed security‑skill files, supports 150+ AI providers (including offline models), and can orchestrate real tools locally or on remote Bolt servers. A TUI, optional web UI, and built‑in HackBrowser complete the workflow.

  • Project
    oomol-lab/open-connector

    OpenConnector is an open‑source gateway that lets AI agents call thousands of SaaS APIs via a single, secure runtime. It handles OAuth/API‑key storage, provides a catalog of ready‑made actions, offers a web console and TypeScript SDK, and can be run locally, on Docker, Cloudflare Workers, Kubernetes, Fly.io, or as a managed OOMOL service.

  • Dispatch
    DeepInfra Integration with Hugging Face Inference Providers

    Hugging Face has added DeepInfra as a supported Inference Provider, enabling serverless access to over 100 models, including DeepSeek V4 and Kimi-K2.6, directly through the Hub and client SDKs.