future-agi/future-agi

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

What it is

Future AGI is a self-hostable, Apache-2.0 platform for building and operating reliable AI agents. It replaces the typical patchwork of separate tools (Langfuse for tracing, Braintrust for evals, Guardrails AI for safety, Helicone for gateway, etc.) with a single integrated system.

The core pitch: agents hallucinate, and Future AGI helps you fix that faster by closing a loop — simulate edge cases → evaluate quality → protect against attacks → monitor live behavior → feed real traces back into optimization.

Key capabilities (six pillars)

  • Simulate — generate thousands of multi-turn conversations with realistic personas, adversarial inputs, and edge cases. Supports text and voice (LiveKit, VAPI, etc.).
  • Evaluate — 50+ metrics under one call: groundedness, hallucination, tool-use correctness, PII, tone, custom rubrics. Uses LLM-as-judge, heuristics, and ML.
  • Protect — 18 built-in scanners (PII, jailbreak, injection) plus 15 vendor adapters (Lakera, Presidio, Llama Guard). Can run inline in the gateway or as a standalone SDK.
  • Monitor — OpenTelemetry-native tracing for 50+ frameworks (LangChain, LlamaIndex, CrewAI, DSPy). Span graphs, latency, cost, live dashboards — zero-config.
  • Agent Command Center — an OpenAI-compatible gateway with 100+ providers, 15 routing strategies, semantic caching, virtual keys, MCP, A2A. Claimed performance: ~29k req/s, P99 ≤ 21 ms with guardrails on.
  • Optimize — six prompt-optimization algorithms (GEPA, PromptWizard, ProTeGi, Bayesian, Meta-Prompt, Random). Production traces feed back as training data.

Architecture

  • Edge: traceAI (OpenTelemetry instrumentation) and Agent Command Center (gateway proxy).
  • Platform: tracer (OTLP ingest), agentic_eval (metrics), simulate (scenario generation), model_hub (LLM routing), accounts (auth, metering).
  • Data: PostgreSQL (metadata), ClickHouse (spans), Redis (state), Temporal (jobs).
  • Interfaces: everything is open — OTLP for traces, OpenAI-compatible HTTP for the gateway, SQL for storage. You can swap components.

Quickstart

  • Requires Docker (2+ vCPUs, 4 GB RAM for standalone).
  • One command: git clone + ./bin/install (or PowerShell script on Windows).
  • After boot, sign in, copy API keys, and send your first trace via pip install fi-instrumentation-otel.
  • SDKs available for Python, TypeScript, Java, C#.

Deployment options

  • Docker Compose (Standalone or Distributed), production overlay, Kubernetes/Helm chart, cloud VMs, and air-gapped/on-prem with FUTURE_AGI_TELEMETRY_DISABLED=true.

Licensing

Apache 2.0 core; individual SDKs are independently licensed (Apache/MIT). The platform is open-source, self-hostable, with a managed cloud option.

Related

  • Project
    hyperspaceai/agi

    A decentralized P2P network where autonomous AI agents collaboratively conduct ML research, pool compute for distributed training and inference, and share discoveries via a global leaderboard.

  • Project
    coze-dev/coze-loop

    A developer-oriented platform for the full lifecycle management of AI agents, providing tools for prompt engineering, automated evaluation, and execution observability.

  • Project
    plurai-ai/intellagent

    A multi-agent framework that stress-tests conversational AI agents by simulating thousands of realistic edge-case interactions to uncover failure points and optimize performance.

  • Project
    zgiai/zgi

    A source-available Agent Runtime platform for building, orchestrating, and operating AI agents and executable workflows with built-in governance and sandboxed execution.

  • Project
    facebookresearch/meta-agents-research-environments

    A research platform for evaluating AI agents in dynamic, realistic environments that require multi-step reasoning and adaptation, featuring the Gaia2 benchmark.