wfzyx/von

What it solves

Von is a non-autoregressive decision model designed to replace large language models (LLMs) for tasks like classification, intent routing, and guardrail validation. It eliminates the high latency, memory overhead (KV cache), and nondeterministic parsing errors associated with token-by-token text generation, providing deterministic, statistically calibrated decisions in under 25ms.

How it works

Built on ModernBERT-Large (395M parameters), Von uses a bidirectional attention head to evaluate input state against specific criteria in a single forward pass. It implements three mathematical primitives for decision-making:

• Choice: Categorical decisions using a normalized probability distribution over mutually exclusive hypotheses.
• Noul: Binary probability verification to estimate if a condition holds true.
• Score: Ordinal continuous rating that computes an expected value across a sequence of severity or quality levels.

To ensure reliability, it is post-trained using Reinforcement Learning with Calibration Distribution (RLCD), which optimizes both accuracy and probabilistic calibration (using a composite Cross-Entropy and Brier Score loss).

Who it’s for

Developers building high-frequency operational pipelines, real-time interactive systems (such as robotics or gaming), and agentic workflows that require fast, calibrated, and deterministic routing and triage.

Highlights

• Ultra-Low Latency: Sub-25ms inference, significantly faster than autoregressive LLMs.
• Non-Autoregressive Parallelism: Evaluates multiple independent questions across a state simultaneously in one pass.
• Calibrated Uncertainty: Output probabilities reflect true predictive confidence via temperature scaling.
• Hardware Agnostic: Optimized for NVIDIA CUDA, AMD ROCm, Apple Silicon MPS, and CPUs.
• SOTA Performance: Outperforms commercial alternatives in real-time ViZDoom arena combat and high-accuracy multi-hop reasoning benchmarks.
• Production-Ready: Includes a Python/TypeScript SDK and a compatible HTTP server (von serve).