Heman10x-NGU/openJev-verdict-2.0

Calibrated 151M Non-Autoregressive Decision Engine beating TypeSafe Jev & Laya on LocalLLaMA/typed-decisions (77.10% acc, 0.0636 Brier, 0.0144 ECE)

What it solves

openJev-verdict-2.0 addresses the challenge of integrating AI into deterministic software workflows. Large language models often produce unstructured text that is difficult for code to parse reliably. This project provides a decision engine that evaluates typed schemas (such as choices or scores) in a non-autoregressive way, allowing software to maintain control over business logic while receiving precise, calibrated semantic classifications.

How it works

The engine uses a 149.6M parameter model based on ModernBERT-base. It processes decisions in a single forward pass, taking 20–25ms. It features a dual-channel calibration system: one channel handles distribution logic (the decision itself), while a second, dedicated confidence head provides highly accurate probability estimates. This allows developers to set thresholds to automatically route ambiguous cases to human operators.

Who it’s for

This tool is designed for developers building enterprise applications—such as financial, security, or customer support systems—who need reliable, high-speed decision-making that runs locally. It is ideal for those who want to avoid cloud inference costs, keep sensitive data on-device, and eliminate the unpredictability often found in standard generative models.

Highlights

  • High Efficiency: Outperforms models 2.8x larger in size with 77.10% accuracy on enterprise decision benchmarks.
  • Local Execution: Runs directly in web browsers via WebGPU, ensuring zero data exfiltration and no cloud costs.
  • Calibrated Confidence: Features a dedicated confidence head with 1.44% error (ECE), enabling predictable human-in-the-loop escalation.
  • Order Robustness: Uses symmetric KL-divergence training to minimize "flip rates" when input options are shuffled, significantly reducing bias compared to standard models.

Related

  • Project
  • Project
  • Project
  • Project