OctaFuse/octafuse-gateway

What it solves

Octafuse Gateway provides a centralized, self-hosted entry point for managing fragmented AI resources. It solves the problem of having to manage multiple API keys, endpoints, and billing schemes across different AI providers (text, image, speech) and agent tools, consolidating them into a single governable infrastructure.

How it works

It acts as an AI proxy that intercepts requests and routes them to various upstream providers based on configurable strategies. It supports multiple protocols (OpenAI, Anthropic, Gemini, DashScope) and integrates agent-specific tools like web search and scraping. The system includes a management backend for configuring providers, routing logic, and a three-tier accounting system (catalog price, cost price, and user price) to track spending and budgets.

Who it’s for

It is designed for individuals building "super individuals" (OPC) or enterprises needing an AI capability hub to distribute AI resources across teams while maintaining strict control over costs, auditing, and access.

Highlights

• Unified AI Entry Point: Consolidates text, image, speech, and agent tools under one BaseUrl.
• Agent-Centric Tooling: Built-in support for web search, deep search, and page scraping with dedicated logging and billing.
• Advanced Routing: Offers strategies including hash affinity (for prompt cache optimization), weighted random, weight priority, and weighted round-robin.
• Enterprise Governance: Features a three-ledger accounting system, time-based pricing multipliers, and user-level model discounts.
• Flexible Deployment: Can be deployed via Docker with Postgres/MySQL or as a serverless application using Cloudflare Workers and D1.