NandhaKishorM/laya

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

What it solves

Traditional systems that use large language models to make decisions (like routing a support ticket) typically generate a sentence and then parse it, which is slow, error-prone, and can produce hallucinations. Laya avoids that entirely: it takes a piece of text and directly outputs a structured answer (a choice from a list, a score, or a yes/no probability) in a single forward pass. This makes decision-making fast (33 ms per question) and reliable, even across 100+ languages.

How it works

Laya uses encoder-only transformer models (ModernBERT and mBERT) that are fine-tuned with a reinforcement learning technique called RLCD, which optimizes against strictly proper scoring rules so the output probabilities are well-calibrated. Instead of generating text token-by-token, it processes the entire input and predicts answers to typed questions in one step. A Router first detects the language and script of the input, then automatically selects the best checkpoint: English, multilingual, or a specialized typed-decisions model. The library supports three question types: choice (pick one category), score (rank on a scale), and noul (give a probability that the answer is yes). It also handles long documents up to 8,192 tokens and can fine-tune on your own domain data for higher accuracy.

Who it's for

Developers building systems that need to classify user messages, support tickets, emails, or any short-to-medium text into structured decisions quickly and reliably — especially in multilingual environments. It's useful for triage, content moderation, intent classification, and as a fast routing layer in agent frameworks like LangChain, LlamaIndex, or CrewAI. It also fits applications that require calibration (knowing the model's confidence) and want to avoid the latency and fragility of text generation.

Highlights

  • Single forward pass: no text generation, no parsing, no hallucination.
  • Extremely fast: 33 ms per question, 7.2 ms per question when batched (on a T4 GPU).
  • Multilingual: supports 100+ languages with a dedicated checkpoint; automatic language detection and routing.
  • Three pretrained checkpoints (English, multilingual, typed-decisions) with a router that picks the right one per request.
  • Fine-tuning on domain-specific decisions can dramatically boost accuracy (example: 0.766 vs 0.362 baseline).
  • Long-context support up to 8,192 tokens with the multilingual model.
  • Broad integration: Python, TypeScript, Rust, and .NET SDKs, plus LangChain, LlamaIndex, CrewAI, and ONNX runtime.
  • Also includes a CLI for quick testing and batch processing.

Related

  • Dispatch
  • Project
  • Project
  • Dispatch
  • Project