ogx-ai/ogx

OGX – Open‑source, OpenAI‑compatible API server

What it is – OGX (formerly Llama Stack) is a self‑hosted API server that mimics the OpenAI REST endpoints (/v1/chat/completions, /v1/embeddings, etc.). It lets you run any LLM—Llama, GPT, Gemini, Mistral, or a custom model—behind the same client‑side code you would use with OpenAI’s cloud service.

Key capabilities

• Standard OpenAI endpoints for chat, completions, embeddings, batches, vector stores, and file handling.
• Responses API that adds server‑side tool‑calling, RAG (file search), and multi‑step orchestration in a single request.
• Skill bundles (/v1alpha/skills) – versioned zip packages that agents can invoke.
• Multi‑SDK support – native routes for Anthropic (/v1/messages) and Google GenAI (/v1alpha/interactions) alongside OpenAI routes.
• Pluggable model providers – works with local Ollama, vLLM, or any managed inference service; the API stays unchanged.
• Open Responses conformance – passes the OpenResponses test suite for standardized agentic responses.

How you use it

1. Install the server (one‑line script or uv pip install ogx).
2. Run uv run ogx go; the process auto‑detects available providers.
3. Point any OpenAI‑compatible client (Python, TypeScript, curl, etc.) at http://localhost:8321/v1 with a dummy API key.
4. Call the usual OpenAI methods, e.g.:

   from openai import OpenAI
   client = OpenAI(base_url="http://localhost:8321/v1", api_key="fake")
   resp = client.chat.completions.create(model="llama-3.3-70b", messages=[{"role":"user","content":"Hello"}])
   

Who it’s for – developers building AI applications who want production‑grade, model‑agnostic serving without rewriting client code, and teams that need built‑in RAG or tool‑calling capabilities.

Documentation & community – Full docs, quick‑start notebook, and API reference are hosted at ogx-ai.github.io. Official Python and TypeScript client SDKs are provided. Community support happens on Discord and weekly community calls.

---

TL;DR – OGX is a drop‑in, open‑source replacement for the OpenAI API that can wrap any LLM and adds extra agentic features (tool calling, RAG, skill bundles) while staying compatible with existing OpenAI, Anthropic, and Google GenAI client libraries.