OpenJev: Browser-Based Decision Modeling and Logit Readout

OpenJev is a local, browser-based implementation designed to reproduce the interface pattern of Jev, a closed-service decision model by TypeSafe. It allows users to compare two distinct methods of extracting decisions from a Large Language Model (LLM): direct logit readout and token-by-token JSON generation.

Direct Logit Readout vs. Token Generation

OpenJev demonstrates that decision-making can be achieved more efficiently by reading the model's internal probabilities (logits) rather than forcing the model to generate text.

Direct Logit Readout (No Decoding)

In this method, the system reads the model's choice logits and normalizes them across a specific set of provided options. This bypasses the decoding process entirely, providing a probability distribution across the allowed choices without the model needing to "write" an answer.

Token-by-Token Generation

This is the traditional LLM approach where the model is prompted to estimate a distribution and write the result as JSON text. This process is significantly slower as it requires the model to generate every token of the JSON structure sequentially.

Supported Models and Performance

OpenJev runs entirely in the browser using WebGPU and quantized GGUF builds via wllama. The weights are sourced from Hugging Face and cached locally; no data leaves the user's browser.

Model	Size	Authored Accuracy	Perturbed Accuracy	TypeSafe Accuracy
Qwen3 0.6B	639 MB	44.0%	52.8%	40.7%
MiniCPM5 2B	1.56 GB	68.6%	69.3%	63.7%
Qwen3.5 4B	3.01 GB	81.3%	76.6%	84.5%
Published Jev	Hosted	—	—	88.3%

Technical Implementation and Limitations

Inference Engine

OpenJev utilizes WebGPU for local execution. However, users have reported compatibility issues with non-Chromium browsers (e.g., Firefox) and certain mobile devices (e.g., iPad Pro), where the lack of a GPU adapter prevents the model from running.

Accuracy and Calibration

Direct readout scores are a softmax over only the displayed option tokens. These are conditional probabilities and are not calibrated confidence scores, nor do they account for every possible answer the model might prefer outside the provided options.

Community Insights and Counterpoints

Discussion among developers suggests that while OpenJev reproduces the interface, it does not reproduce the proprietary training or the specific model used by the original Jev service.

On the "Moat" of Decision Models

Some developers argue that the core innovation of Jev is not the API but the specific training and the model itself. One contributor noted that the underlying mechanism—restricting output to a specific set of tokens—is a known technique:

"The last step of an LLM is to take a softmax of the predictions and then generating a token from that. But there was tooling that would just generate all allowed next tokens from a grammar... zeroing all the ones not allowed and then picking the best among the allowed tokens."

Alternative Implementations

Community members have pointed to other ways to achieve similar results, such as a vLLM patch that turns DiffusionGemma into a Jev-like model, which reportedly matches Jev's evaluation performance more closely than the small open-weight models used in OpenJev.

User Experience Critiques

Users have criticized the OpenJev demo site for its visual design, describing it as "vibecoded" or LLM-generated, which some found distracting from the technical experiment. Others noted that the site lacks a clear explanation of the purpose of the technology for non-technical users.