malevrigns/agent-jev

What it solves

AgentJev solves the inefficiency and unreliability of using large generative LLMs for simple decision-making tasks (like gating, routing, or scoring) within an agent loop. Instead of asking a large model to write a reasoned response and then parsing that text into a boolean or category, AgentJev provides a direct probability distribution over a set of predefined options in a single forward pass, eliminating the need for token decoding and JSON parsing.

How it works

Built on a Qwen3-0.6B backbone with the language-model head removed, AgentJev uses a specialized candidate head to score options. It takes a state (e.g., a diff, stack trace, or ticket) and a typed question, then returns a probability distribution for one of three primitives:

• Boolean: A true/false proposition.
• Choice: A selection from 2-255 provided options.
• Score: A value on an ordered rubric (2-10 levels).

To optimize performance, it employs a shared-prefix caching mechanism that encodes the state once and branches for each candidate option, significantly reducing redundant computations.

Who it’s for

It is designed for developers building AI agents, triage bots, or complex workflows where a small, fast "reflex" model is needed to make routing or safety decisions without the overhead of a full generative turn.

Highlights

• Zero Decoded Tokens: Returns distributions directly from hidden states, removing the risk of malformed text outputs.
• High Efficiency: Uses KV prefix reuse to achieve up to 2x faster performance on wide candidate sets compared to non-causal encoders.
• Typed Primitives: Provides structured outputs for Booleans, Choices, and Scores, including confidence margins and expected values.
• Integrated Gating: Includes a pre-tool-use hook for tools like Claude Code to block unsafe actions based on model scores.