OpenJev Brings Local Decision Models to the Browser

OpenJev enables on‑device decision models in the browser

OpenJev is a web‑based demo that runs compact LLMs (e.g., MiniCPM5 2B, Qwen3 0.6B) locally in the browser, allowing users to compare two inference methods: direct logits readout and token‑by‑token JSON generation. The project shows that structured decision outputs can be produced without sending data to a server.

Direct logits readout is faster but uncalibrated

The read logits mode reads the model’s raw probability distribution for the supplied options and normalizes it across those tokens only. This avoids any token decoding step, so the wall‑time is limited to model evaluation and a small post‑processing step. The demo reports that on a desktop GPU the direct method finishes in roughly half the time of the generation method. However, the probabilities are conditional on the displayed options and are not calibrated confidence scores.

"Conditional probabilities. Direct scores are a softmax over only the displayed option tokens. They are not calibrated confidence and do not include every answer the model might prefer." – OpenJev documentation

JSON generation produces explicit token streams

In the JSON generation mode the model is prompted to emit a JSON object that lists each option together with its probability. The demo streams each token as it is generated, letting users observe the incremental construction of the JSON. This method incurs the full decoding latency of the model, typically taking about twice as long as the direct readout on the same hardware.

"Ask the model to estimate the same displayed‑option distribution and write it as JSON. Watch every token arrive." – OpenJev documentation

Model selection and performance trade‑offs

OpenJev ships with four model configurations:

Model	Size	Authored accuracy	Perturbed accuracy	TypeSafe accuracy
Qwen3 0.6B	639 MB	44.0 %	52.8 %	40.7 %
MiniCPM5 2B	1.56 GB	68.6 %	69.3 %	63.7 %
Qwen3.5 4B	3.01 GB	81.3 %	76.6 %	84.5 %
Published Jev (hosted)	–	–	–	88.3 %

Authored refers to the model’s performance on the original training distribution, while Perturbed measures robustness to input variations. TypeSafe reflects the accuracy of the same 102‑row subset used by the original Jev service. The hosted Jev service still outperforms the open models on this benchmark.

Technical implementation details

OpenJev uses WebGPU (when available) and the wllama runtime to load GGUF‑quantized weights directly in the browser. The weights are fetched from Hugging Face and cached locally; no request ever leaves the page. The demo performs the following steps:

Download & cache the model file (e.g., 1.56 GB for MiniCPM5). The first load can take several minutes.
Warm‑up by compiling GPU passes for both inference paths.
Prompt preparation that concatenates a shared state with the user‑provided question and options.
Direct logits: run the model once, extract the logits for the option tokens, and normalize.
JSON generation: run the model again, this time prompting it to emit a JSON object token‑by‑token.

"The methods run sequentially on the same loaded model so they do not contend for one GPU. Direct runs first, then generation." – OpenJev documentation

The demo’s timing uses performance.now() to capture setup, warm‑up, input preparation, execution, and generation completion, providing a transparent measurement of real‑world latency.

Community observations and criticisms

Usability concerns – Several commenters note the UI feels cluttered and the site’s purpose is unclear, especially for newcomers unfamiliar with Jev.
Performance on low‑end devices – Users report slow loads on iPad Pro and phones, even with the smallest 0.6 B model, suggesting that the demo still demands substantial GPU resources.
Legal and naming issues – Some point out that “Jev” is a trademark and the open implementation may cause confusion about its relationship to the proprietary service.
Technical comparisons – A few participants measured the model’s decision outputs on simple binary questions and observed that direct logits often produce highly skewed probabilities, whereas the generation method defaults to a near‑uniform distribution, effectively acting as a coin flip.
Potential for open‑source alternatives – Multiple comments reference ongoing efforts to integrate Jev‑style decision APIs into open‑source inference engines (e.g., vLLM patches, TypeSafe adapters), indicating a broader interest in replicating Jev’s speed and deterministic token restriction.

What OpenJev tells us about the future of on‑device LLM utilities

OpenJev demonstrates that structured decision outputs—probability distributions over a fixed set of options—can be produced locally with modest hardware, without the need for a backend service. By exposing both a fast logits‑only path and a more transparent JSON generation path, the demo highlights the trade‑off between latency and interpretability. While the current open models lag behind the proprietary Jev service in accuracy, the approach validates the feasibility of client‑side decision models and may inspire further optimizations in quantization, caching, and inference engine design.