vinilana/jev-gateway

An easy way to use jev with your coding agent for tool calling reasoning

What it solves

jev-gateway reduces the cost and latency of coding agents by offloading the decision of which tool to call to a specialized, fast decision model called Jev. Instead of using an expensive reasoning model to determine the next tool, the gateway intercepts the request, asks Jev for the best tool, and then steers the main LLM to that tool, while leaving all other processing to the original model.

How it works

The project acts as a local proxy (gateway) that sits between a coding agent and its LLM provider. When an agent is about to make a tool call, the gateway asks the Jev API (available via TypeSafe, OpenRouter, Vercel AI Gateway, or OpenCode) to select the appropriate tool.

Depending on Jev's confidence, the gateway takes different actions:

  • Forced: Jev picks a tool, and the LLM is forced to use it, filling in the remaining arguments.
  • Direct: Jev picks a tool and all arguments are fixed (enums/booleans), allowing the gateway to answer without calling the LLM at all.
  • None: Jev decides no tool is needed.
  • Passthrough: If Jev is unconfident, slow, or unavailable, the request is forwarded to the LLM untouched to ensure the agent never fails.

Who it’s for

Developers using AI coding agents such as Codex, Claude Code, OpenCode, Kilo, or Devin who want to optimize their tool-calling efficiency and reduce reliance on expensive reasoning models for simple routing decisions.

Highlights

  • Out-of-the-box support: Works with Codex, Claude Code, OpenCode, Kilo, Gemini, and Devin.
  • Zero-config integration: Provides jev- launcher commands (e.g., jev-codex) that wrap existing agents without modifying their local configuration files.
  • Monitoring Dashboard: Includes a web-based dashboard to track routing decisions, latency, token usage, and cost comparisons against a baseline.
  • Fail-safe design: Requests automatically fall back to the original LLM if the gateway or Jev API encounters an error.
  • Flexible Provider Support: Can connect to Jev via multiple providers including TypeSafe, OpenRouter, Vercel, and OpenCode.

Related

  • Project
    thushan/olla

    A high-performance LLM load balancer and proxy that intelligently routes requests across self-hosted inference nodes with automatic failover and health monitoring.

  • Project
    shlokkhemani/rabbithole

    Rabbithole is a locally‑hosted web app that creates a branching canvas of documents answered by any LLM (Claude Code, Codex, OpenAI‑compatible endpoints). It stores everything on your machine, offers a bridge to Claude Code/Codex, and provides an MCP server for tool‑call integration.

  • Project
    vllm-project/agentic-api

    A Rust-native stateful API gateway for vLLM that moves conversation state and tool-execution loops from the client to the server.

  • Project
    AliAkhtari78/SpotifyScraper

    A Python library for extracting public Spotify metadata without an API key, featuring an MCP server for AI agents and support for lyrics and transcripts.