Floe-Labs/floe-guard

What is floe‑guard?

floe‑guard is a lightweight library (available for both Python and TypeScript) that lets you track and enforce a dollar budget for every AI call your application makes – whether the call is to a large‑language‑model, a speech‑to‑text service, a text‑to‑speech service, a telephony provider, or any other AI‑related vendor.

Why it matters

• AI agents often call many services (OpenAI, Anthropic, Gemini, voice STT/TTS, telephony, etc.).
• Those calls can add up quickly, and traditional usage limits (max_tokens, max_rpm) don’t protect you from a runaway spend.
• floe‑guard records the real USD cost of each call as soon as it finishes and can stop the next call before it would exceed a budget you set.

Core concepts

Concept	What it does
BudgetGuard	Main class you instantiate with a dollar ceiling (e.g. BudgetGuard(limit_usd=5.00)). It provides check() (pre‑call) and record() (post‑call) methods.
Hard‑stop	If check() predicts the next call would push you over the ceiling, it raises BudgetExceeded and the call never reaches the vendor.
Live ledger	All spend is kept in‑process (no account, no network) and can be exported to JSONL. Optionally you can push the ledger to Floe’s hosted service for a “Coverage Score” and a 7‑day history.
Adapters	Ready‑made wrappers for popular stacks: OpenAI, Anthropic, Gemini, CrewAI, LiteLLM, LangChain, LangGraph, Vercel AI SDK, plus voice adapters for Pipecat, LiveKit, Vapi, Retell. They automatically call check() before the request and record() after.
Rate cards	The library ships with a bundled cost map (public list prices snapshot). You can override any price with your own FLOE_RATE_CARD JSON so the guard reflects negotiated contracts instead of list rates.
Tool reservation	For paid tools you can call reserve_tool() before the tool runs and settle_tool() after, guaranteeing atomic budget checks even when many calls happen in parallel.
Persistence	A SqliteStore lets multiple processes share the same daily budget (window="utc-day").

How you use it (Python example)

from floe_guard import BudgetGuard

guard = BudgetGuard(limit_usd=5.00)   # $5 ceiling

guard.check()                         # raises if next call would exceed $5
response = client.chat.completions.create(  # your normal LLM call
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)
# After the call finishes, record the spend
guard.record(
    model="gpt-4o",
    prompt_tokens=response.usage.prompt_tokens,
    completion_tokens=response.usage.completion_tokens,
)

If the call would push the total over $5, guard.check() throws BudgetExceeded and the request never hits OpenAI.

Voice‑call support

Voice pipelines involve several legs (STT → LLM → TTS → telephony). floe‑guard provides per‑turn adapters that reserve budget before each turn and settle after the turn finishes, ensuring the whole voice session stays under the same dollar ceiling.

No‑account, no‑telemetry mode

• By default everything runs locally – no API key, no signup, no outbound telemetry.
• You can optionally connect a free Floe account (one key) to get a Coverage Score and a 7‑day spend history, but the core enforcement works entirely offline.

Quick demos you can run immediately

• pip install floe-guard && floe-guard demo – shows a stub LLM loop that stops before a $0.10 ceiling.
• floe-guard estimate gpt-4o --calls 1000 --tokens-in 800 --tokens-out 300 – estimates cost for a workload using the bundled map.
• Voice‑call cost demo (examples/voice_call_cost_livekit.py) – prints a per‑leg cost breakdown without any network calls.

Who might need it?

• Developers building autonomous agents (CrewAI, LangChain, etc.) that could get stuck in loops.
• Teams that bill clients per‑usage and need to guarantee they never over‑charge.
• Anyone who wants a pre‑emptive spend guard rather than a post‑mortem usage report.

---

Bottom line: floe‑guard is a practical, open‑source guardrail that measures the actual dollar cost of every AI‑related API call and can stop execution before you exceed a budget you define. It works offline, integrates with the major LLM and voice toolkits, and lets you plug in your own negotiated rates.