Laya: An Open-Source System 1 Decision Engine

Laya is a non-autoregressive decision engine designed to replace generative LLMs for simple, structured reflex decisions. By utilizing bidirectional encoders instead of autoregressive token generation, Laya delivers calibrated probability predictions over structured schemas in approximately 32.8 milliseconds on a single GPU, making it significantly faster and more cost-effective than generative alternatives.

The System 1 Approach to AI Decisions

Modern AI pipelines often suffer from a bottleneck where large generative LLMs (System 2) are used for tasks that only require a reflex-like response (System 1). Tasks such as ticket routing, spam detection, and jailbreak identification do not require text generation but rather a specific label or probability.

Using a generative LLM for these tasks introduces several inefficiencies:

Latency: Waiting for tokens to stream (500ms to 2,000ms).
Cost: High inference costs for 8B+ parameter models.
Reliability: The need for regex or JSON parsers to extract labels from free-form text.
Calibration: LLMs often hallucinate confidence scores (e.g., outputting "confidence: 0.95" without mathematical grounding).

Laya addresses this by providing instant, calibrated probabilities in a single forward pass, eliminating the possibility of hallucinations or malformed JSON outputs.

Decision Primitives and Architecture

Laya evaluates typed questions over any state (text, email, or JSON) using three core primitives:

Choice: Selects one option from a dictionary of criteria, returning the selected key, the probability distribution, and a calibrated confidence score.
Score: Places the state on an ordinal rubric (e.g., 0 to 3), returning the expected level and distribution.
Noul: A boolean question returning a calibrated probability $P(\text{true})$ from 0.0 to 1.0.

Model Checkpoints

Laya is distributed via a bundled hub on Hugging Face, offering three specialized checkpoints:

Checkpoint	Backbone Encoder	Params	Context	Primary Strength
`laya`	ModernBERT-large	421M	512	English classification, guardrails, email triage
`laya-multilingual`	mmBERT-base	322M	1024	100+ languages, cross-lingual NLI
`laya-typed-decisions`	ModernBERT-large	421M	1024	Agent observability, invoice processing, security alerts

Multilingual Routing and Script Detection

A critical finding in Laya's development was that English-centric models often report high confidence even when they cannot read the input script (e.g., reporting 95% confidence on Khmer text while having 0% accuracy). Because confidence gating is unreliable for unsupported scripts, Laya implements a sub-millisecond pure Python Router.

This router inspects Unicode scripts across 22 alphabets and analyzes Latin stopword distributions to route traffic to the appropriate model before the forward pass occurs. Detection overhead is negligible, typically ranging from 0.09ms for English to 0.73ms for large JSON documents.

Performance Comparison: Laya vs. TypeSafe Jev

Laya is positioned as an open-source alternative to TypeSafe Jev. Benchmarks indicate significant advantages in speed and cost, though some trade-offs exist in zero-shot capabilities.

Metric	TypeSafe Jev 1.13.0	Laya (Routed)	Delta
Latency P50 (1 Question)	236–276 ms	32.8 ms	7.8x faster
Latency P50 (10 Qs Batched)	~1,500 ms	72.3 ms	20x faster
Calibration Error (ECE)	0.246	0.081	3x better calibration
Cost per 1M tokens	$0.042	$0.00	Free (Apache 2.0)
Weights & Code	Closed API	Open-source	Self-hostable

Engineering Limitations and Requirements

Laya is a foundation model for specialization rather than an omniscient zero-shot oracle. Users should be aware of the following constraints:

Fine-tuning Necessity: Out-of-the-box base models may score near random (~0.35) on specific benchmarks like typed-decisions. High accuracy (0.766) is achieved through fine-tuning on the training split.
Choice Budget: Performance degrades when choice schemas exceed 20 options due to token budget constraints in the head length.
Context Window: Checkpoints are limited to 512–1024 tokens, which is significantly smaller than the 32k context window reported for Jev.
Temperature Calibration: To reduce expected calibration error from 0.466 to 0.081, a scalar temperature per question type should be fitted to the domain distribution.

Community Insights and Counterpoints

Discussion among technical users highlights a divide between "model" and "product." While Laya provides the weights and architecture, critics argue that Jev's value lies in its zero-shot performance and ease of deployment via API.

"The key to Jev's success is that it works without fine tuning... the ability to knock out any arbitrary classification problem in minutes instead of in a week is a big deal."

Other users noted that Laya's approach is a return to "conventional ML," essentially utilizing BERT-style architectures with updated training methods (RLCD) to solve classification tasks more efficiently than generative LLMs.

Quickstart Implementation

Laya can be installed via pip (pip install laya>=0.3.3) and used with a Router to handle multi-schema decisions:

from laya import Router

router = Router(preload=True)

questions = {
    "queue": {
        "type": "choice",
        "instructions": "Which engineering queue owns this ticket?",
        "criteria": {
            "infrastructure": "server outages, network downtime",
            "billing": "refunds, SLA credits"
        }
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this ticket?",
        "criteria": ["low", "medium", "high", "critical"]
    }
}

res = router.predict({"body": "API is down!"}, questions)