antoinezambelli/forge

What it solves

Forge is a reliability layer designed to make tool-calling (function calling) more dependable, especially for self-hosted local LLMs. It prevents common failures where models emit malformed JSON, call non-existent tools, or fail to follow a specific sequence of steps, which often causes agentic workflows to crash or loop indefinitely.

How it works

Forge acts as a guardrail system that can be integrated in three ways:

1. Proxy Server: A drop-in sidecar that sits between an existing client (like Aider or Claude Code) and a model server. It transparently intercepts requests and applies guardrails without requiring code changes to the client.
2. WorkflowRunner: A high-level framework for defining tools and structured agent loops, managing the full lifecycle from system prompts to context compaction.
3. Guardrails Middleware: A composable set of tools that can be plugged into custom orchestration loops to validate responses and rescue malformed calls.

Key reliability mechanisms include rescue parsing (extracting tool calls from non-standard formats like XML or code fences), response validation (checking tool names and shapes), and automatic retry loops with corrective nudges.

Who it’s for

Developers building AI agents or using existing coding assistants who rely on local LLMs and want to increase the success rate of tool-calling without switching to larger, more expensive frontier models.

Highlights

• High Reliability Lift: Increases local 8B model success rates from single digits to 84% on its evaluation suite.
• Broad Backend Support: Works with llama-server, Ollama, vLLM, Llamafile, and Anthropic.
• Zero-Rewrite Integration: The proxy mode allows existing OpenAI-compatible tools to benefit from guardrails transparently.
• Context Management: Includes strategies like tiered compaction to manage token budgets efficiently.
• Priority Access: The SlotWorker allows multiple specialist workflows to share a single GPU slot via priority queuing.