razorback16/openjev

What it solves

OpenJev provides a fast, calibrated way to get typed decisions (yes/no, multiple choice, or numerical scores) from open-source AI models. Unlike traditional LLMs that generate text which must then be parsed, OpenJev reads probabilities directly from the model's output distribution, ensuring that answers always adhere to a strict schema and cannot go "off-schema."

How it works

OpenJev primarily utilizes discrete diffusion models like DiffusionGemma. Instead of generating text token-by-token from left to right, it uses a read-only pass to denoise a "canvas" where only the answer slots are masked. The model's probability distribution for the labels in those slots (e.g., "yes"/"no" or "A"/"B") becomes the answer.

It also supports smaller bidirectional encoder models (like Laya and Verdict) that use classification heads to provide probability distributions over options in a single forward pass.

Who it’s for

It is designed for developers who need high-speed, structured decisions from a model—such as classifying customer intent, determining urgency, or scoring sentiment—without the latency or reliability issues associated with text parsing.

Highlights

• Strictly Typed Outputs: Supports noul (yes/no), choice (multiple choice), and score (numerical levels).
• High Performance: Returns answers in tens of milliseconds for short states.
• Calibrated Confidence: Provides a confidence score based on the model's own probability distribution rather than a self-reported number.
• Multimodal Support: Can process both text and images to make decisions.
• Flexible Deployment: Runs via vLLM on NVIDIA GPUs or via MLX on Apple silicon.