Laya Open-Source System 1 Decision Model Beats TypeSafe Jev

TL;DR

Laya is an open‑source, bidirectional‑encoder decision model that makes reflex‑type predictions in 30–35 ms, offers calibrated probabilities, supports over 100 languages, and outperforms TypeSafe AI’s closed‑source Jev by 7–8× in latency while being free to self‑host.

1. System 1 vs. System 2 – Why a Fast Decision Model Matters

Takeaway: Using a generative LLM for simple classification or routing tasks is wasteful; a dedicated System 1 model provides instant, calibrated decisions without hallucination.

  • Generative LLMs need 500 ms–2 s per request, consume token‑based API fees, and require post‑processing to extract structured labels.
  • LLM‑generated confidence scores are uncalibrated token predictions, not mathematically grounded probabilities.
  • Laya implements a human‑like System 1: reflexive decisions in ~33 ms on a single GPU, with honest probability outputs.

2. Decision Primitives – The Core API

Takeaway: Laya exposes three deterministic primitives that return only numbers, eliminating any chance of malformed JSON or hallucinated text.

Primitive Purpose Output Guarantees
choice Select one option from a dictionary selected_key, full probability distribution, calibrated confidence Probabilities sum to 1, confidence reflects true likelihood
score Place input on an ordinal rubric (0, 1, 2, …) Expected level, distribution over ranks, confidence Monotonic calibration across ranks
noul Binary true/false query P(true) in [0.0, 1.0]; P(false)=1-P(true) Exact complementarity, no contradictory outputs

Because the model never emits free‑form text, schema violations are impossible.

3. Model Checkpoints & Hub Architecture

Takeaway: Laya ships three specialized checkpoints in a single Hugging Face repository, and the SDK can download only the needed subfolder to save bandwidth.

Checkpoint Encoder Params Context Strength
convaiinnovations/laya ModernBERT‑large 421 M 512 tokens English classification, guardrails, email triage
convaiinnovations/laya-multilingual mmBERT‑base (256 k vocab) 322 M 1024 tokens (up to 8 k) 100+ languages, 2.2× faster, cross‑lingual NLI
convaiinnovations/laya-typed-decisions ModernBERT‑large 421 M 1024 tokens Agent observability, invoice processing, security alerts (0.766 acc)

Selective subfolder download (Python example):

import laya
# English only (≈808 MB)
agent_en = laya.load("convaiinnovations/laya")
# Multilingual only (≈647 MB)
agent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual")

4. Language Routing – Preventing Silent Failures

Takeaway: Confidence scores do not reveal script‑reading failures; routing must happen before inference.

  • On the MASSIVE benchmark (51 languages, 20‑way classification) the English‑only checkpoint achieved 0.000 accuracy on Khmer while reporting 0.952 mean confidence.
  • Across all non‑Latin scripts, mean confidence stayed above 0.885, even when accuracy dropped to random‑guess levels.
  • Laya’s built‑in Router inspects Unicode scripts in < 1 ms and dispatches to the appropriate checkpoint, eliminating the need for post‑hoc confidence gating.
from laya import Router
router = Router(preload=True)
res_en = router.predict({"body": "I was charged twice, please refund."}, questions)
res_hi = router.predict({"body": "मुझसे दो बार शुल्क लिया गया..."}, questions)
  • Routing overhead is < 2 % of total latency (0.09 ms for Latin, 0.54 ms for Devanagari, 0.73 ms for large JSON).

5. Head‑to‑Head Benchmark vs. TypeSafe Jev

Takeaway: Laya consistently outperforms Jev on accuracy, calibration, and latency while being free and open‑source.

Metric TypeSafe Jev 1.13.0 Laya (Routed) Δ
Typed‑decisions (2 k decisions) 0.727 0.766 +3.9 %
AG News (4‑label) 0.910 0.950 +4.0 %
DAIR Emotion (6‑label) 0.480 (Brier 0.846) 0.595 +11.5 %
Expected Calibration Error (ECE) 0.246 0.081 3× better
P50 latency, single query 236‑276 ms 32.8 ms 7.8× faster
P50 latency, 10‑query batch ~1.5 s (serial) 72.3 ms (7.2 ms/q) 20× faster
Languages with >3× random accuracy – (no public data) 45 / 51 Global coverage
Cost per 1 M tokens $0.042 (metered API) $0.00 (self‑hosted) Free
Weights & code Closed, proprietary Apache 2.0 safetensors Open

Real‑world workflow results (selected):

  • Email spam (Enron) – 0.993 acc, 0.013 ECE
  • Phishing detection – 0.980 acc, 0.012 ECE
  • LLM guardrail / jailbreak detection – 0.755 acc; with 50 % selective coverage accuracy rises to 0.931
  • RAG passage relevance – 0.657 acc in a single forward pass
  • 10‑way ticket routing – 0.522 acc

6. Honest Limitations – When Laya Falls Short

Takeaway: Laya excels on small‑to‑medium choice sets and after fine‑tuning; it is not a universal zero‑shot classifier.

  1. Choice scalability: Accuracy drops sharply beyond ~20 options (e.g., 0.425 on Banking77 with 77 labels vs. Jev’s 0.870). The model’s head can only allocate ~3‑4 tokens per candidate, limiting discrimination.
  2. Zero‑shot performance: Base checkpoints score ~0.35 on the typed‑decisions benchmark (near random). The reported 0.766 figure requires fine‑tuning on the benchmark’s training split.
  3. Temperature calibration: Raw logits need a scalar temperature per question type; applying this reduces calibration error from 0.466 to 0.081.

Recommendation: Use Laya as a fast foundation model, then fine‑tune on domain‑specific data and calibrate temperature for production use.

7. Quickstart – Running Laya in 30 seconds

Takeaway: Installing the PyPI package and invoking the router yields multi‑schema decisions in a single forward pass.

pip install laya>=0.3.3

import laya
from laya import Router
router = Router(preload=True)

ticket = {
    "ticket_id": "TCK-8821",
    "subject": "System downtime and billing dispute",
    "body": "Our production API has been failing since 6 AM..."
}

questions = {
    "queue": {
        "type": "choice",
        "instructions": "Which engineering queue owns this ticket?",
        "criteria": {
            "infrastructure": "server outages, network downtime",
            "billing": "refunds, SLA credits",
            "security": "breaches, vulnerabilities",
            "support": "general inquiries"
        }
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this ticket?",
        "criteria": ["low", "medium", "high", "critical"]
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the customer threaten to cancel?"
    }
}

res = router.predict(ticket, questions)
print("Model used:", res["routing"]["model"])   # → english
print("Queue:", res["answers"]["queue"]["choice"])   # → infrastructure (confidence 0.96)
print("Urgency score:", res["answers"]["urgency"]["score"])   # → 2.87/3.0
print("Churn risk:", f"{res['answers']['churn_risk']['noul']:.1%}")   # → 91.4%

The entire process, including model download (≈800 MB), completes in under a minute on a single GPU.

8. Community Resources

9. Community Insight – What Others Are Saying

"It’s a bit faster and cheaper than LLMs, but essentially it’s BERT with more data. The value is in a one‑shot classifier that can handle many questions at once." – @Oras

"Jev’s branding made the concept go viral, but the underlying idea—non‑autoregressive decision models—has existed for years. Open‑source versions like Laya prove the tech is reproducible." – @kamranjon

"The routing hack is necessary because confidence scores stay high even when the model can’t read the script. Pre‑loading models eliminates the 7‑10 s cold‑swap penalty." – @sandos

"Fine‑tuning is required for good performance; the base model is near random on typed‑decisions. Treat Laya as a fast foundation, not a plug‑and‑play oracle." – @cjalmeida

"Running on a CPU is borderline; the 400 M‑parameter checkpoints need a GPU for the advertised sub‑35 ms latency, but the code is CPU‑compatible for batch‑size 1 inference at ~200 ms. – @tarruda

Conclusion

Laya demonstrates that many high‑throughput AI workloads do not need large autoregressive LLMs. By framing decision‑making as a calibrated, non‑generative prediction problem, Laya achieves 6–8× lower latency, zero hallucination, and full multilingual coverage—all under an Apache 2.0 license. The trade‑off is the need for fine‑tuning on domain data and a modest limitation on the size of choice sets. For organizations that can host their own GPU, Laya offers a production‑ready, cost‑free alternative to proprietary “System 1” services like TypeSafe AI’s Jev.

Sources

Related

  • Dispatch
    TypeSafe AI Jev System One Model: Fast, Structured Decision‑Making AI

    TypeSafe AI’s Jev, the first System One model, delivers frontier‑level intelligence for structured decision tasks at 40‑200× lower latency and 100‑500× lower cost than comparable LLMs, while guaranteeing type‑safe outputs and calibrated confidence scores.

  • Project
    mizorewww/laya-mlx

    Laya‑MLX is a Python library that runs ConvAI’s Laya LLMs locally on Apple‑Silicon via the MLX runtime. It provides ultra‑low‑latency *typed‑decision* inference (choice, score, yes/no) without PyTorch or cloud services, includes a router for multilingual/English models, optional compilation, confidence gating, and CLI tools such as a snake‑game demo.

  • Dispatch
    OpenJev: Browser-Based Decision Modeling and Logit Readout

    OpenJev is a local, browser-based experiment that demonstrates the performance difference between direct logit readout for decision-making and traditional token generation using open-weight models.

  • Dispatch
    Laya CoreML Demo on Apple Silicon M4 – Offline Snake Game Running via CoreML

    A community member demonstrated Laya running offline on a Mac M4 using CoreML, showing low‑memory, neural‑engine‑only inference for a Snake game demo.