ThinkWatchProject/ThinkWatch

ThinkWatch – A Self‑Hosted AI & MCP Gateway

What it does

• Acts as a single, organization‑wide gateway for every LLM request (OpenAI, Anthropic, Gemini, Azure, Bedrock, …) and for MCP tool calls (GitHub, Notion, Slack, etc.).
• Every request is authenticated against the company’s identity provider (OIDC), checked against per‑user / per‑role rate limits and budgets, inspected by three built‑in security guards, priced, and finally logged to an audit store.
• Provides a management console (web UI on port 3001) where admins configure providers, roles, limits, budgets, security rules and view audit data.

Key components

Component	Role
Gateway (port 3000)	HTTP endpoint that speaks the OpenAI, Anthropic and Gemini APIs. It translates incoming calls to the appropriate upstream provider, applies rate‑limits, budgets, security‑guard transformations and records the event.
Console (port 3001)	Admin UI + admin API for configuring providers, OIDC, roles, limits, budgets, security guards and viewing audit logs.
ThinkWatch Core crates	Re‑usable Rust libraries (tw‑dialect, tw‑guard, tw‑breaker, tw‑bedrock) that handle format conversion, guard logic, circuit‑breaker routing and Amazon Bedrock signing.
Storage	PostgreSQL for configuration, ClickHouse for immutable audit logs, Redis (single‑node or cluster) for caching and rate‑limit counters.
Deployment	Docker‑Compose for quick local dev, Helm chart for Kubernetes production deployments.

Why it matters

• Security‑first gateway – Secrets, personal data, and dangerous commands are automatically redacted or blocked before they ever leave the organization. Custom guard rules let security teams enforce policies that match their compliance needs.
• Unified billing & cost control – Requests are tagged with model, provider and cost‑center; the system produces CSV charge‑back reports and forecasts, helping finance teams keep AI spend under control.
• Per‑user tool credentials – When a user links a GitHub, Notion, Slack, etc. account, the gateway uses that user’s token when calling the upstream tool, so the upstream audit log shows the real actor.
• Flexible routing – Traffic can be weighted, routed by latency or health, and a circuit‑breaker removes failing upstreams automatically.

Getting started (quick dev flow)

make infra               # spin up Postgres, ClickHouse, Redis, etc.
make dev-secrets          # generate .env with random secrets
make dev-backend          # start gateway (3000) and console (3001)
cd web && pnpm install && pnpm dev   # run the React admin UI
# Open http://localhost:5173/setup to finish initial configuration

For production you can run make deploy (Docker‑Compose) or make helm-deploy (Kubernetes).

Who should use it

• Enterprises that want a bastion‑host‑style control plane for all AI model usage and tool integrations.
• Teams needing strict data‑privacy guards, per‑user credential handling, and detailed cost accounting.
• Organizations that already run their own identity provider (Okta, Azure AD, Zitadel, etc.) and want seamless SSO for AI workloads.

License & commercial model

• Source‑available under the Business Source License 1.1. Free for non‑production and for production up to 10 M billable tokens or 10 k MCP tool calls per month. Above those thresholds a commercial license is required.

---

All information above is taken directly from the repository’s README.