Aleph Alpha Kolibri Release: A Sovereign Open-Weight Model for Regulated Industries

Aleph Alpha has released Kolibri, an open-weight Mixture-of-Experts (MoE) Transformer designed for sovereign, mission-critical applications in regulated sectors such as public administration, aerospace, and industrial manufacturing. The model is released under the Apache 2.0 license and is available on Hugging Face.

Model Architecture and Efficiency

Kolibri is designed to optimize the trade-off between high-level capability and serving costs, positioning itself on the "Pareto frontier" for quality versus throughput in both English and German.

Parameter Count: 78.1B total parameters, with approximately 3.46B active parameters per token.
Context Window: Supports context lengths up to 1 million tokens (trained up to 256k).
Attention Mechanism: To maintain efficiency, 40 of the 50 layers use a sliding window (512 tokens), while only 10 layers process the full context.
MoE Design: The model utilizes 384 smaller experts rather than a few large ones, which Aleph Alpha reports improved performance in their internal tests.

Bilingual Specialization and Tokenization

Unlike models that are primarily English-centric and later adapted for other languages, Kolibri is bilingual by design.

German-Centric Training

Aleph Alpha focused on organic German data to avoid the "cultural fingerprint" and translation errors associated with machine-translated corpora. German data accounts for 21.3% of the pre-training tokens (approximately 4.3T tokens). This was achieved through:

Custom Curation: Retuning filtering parameters for German administrative prose to prevent the accidental removal of official registers.
LLM Rephrasing: Using LLMs to rewrite organic German documents into different styles (e.g., encyclopedia entries), multiplying the utility of scarce high-quality data.
Minimal Translation: Translation was used sparingly (6% overall) to preserve authentic cultural and institutional context.

UniBPE Tokenizer

Kolibri introduces UniBPE, a tokenizer that combines the bottom-up approach of BPE with the Unigram training objective. This method better respects the morphology of the German language, particularly its compound word structure, resulting in higher compression (fewer tokens per word) compared to standard BPE or other frontier model tokenizers.

Grounding and Hallucination Reduction

To meet the requirements of regulated industries, Kolibri is trained specifically to abstain from answering when the provided context does not contain the necessary information.

The Merlin-Arthur Protocol

Aleph Alpha employs a proprietary Merlin-Arthur protocol to generate synthetic negative examples. This process involves three roles:

Arthur: The model being trained.
Merlin: Generates contexts that increase the probability of a correct answer.
Morgana: Strips evidence from documents to lure the model into hallucinating.

By training against Morgana's redacted contexts, Kolibri learns that abstention is the only correct strategy when evidence is missing. This has resulted in a significant increase in the non-hallucination rate on the AA-Omniscience benchmark (44% for Kolibri vs 14.8% for Kolibri Origin).

The "Model Factory" Pipeline

Kolibri was developed using a "Model Training as Code" approach, allowing Aleph Alpha to iterate from the previous version, Kolibri Origin, in just three months.

Automation: The pipeline is fully automated via GitHub Actions; every code change triggers a small end-to-end run to validate stability.
Robustness: During the 21-day pre-training run on 768 B200 GPUs, the pipeline automatically handled 38 unplanned hardware or connection interruptions without manual intervention.
Training Scale: Kolibri was trained on nearly 24T tokens total, including a three-stage process: pre-training (16k sequence length), mid-training (64k), and long-context adaptation (256k).

Performance and Benchmarks

Kolibri demonstrates competitive performance across math, coding, and agentic tasks, often matching models with up to four times its active parameter count.

Benchmark	Kolibri	Kolibri Origin	Nemotron 3 Super (120B-A12B)	Mistral Small 4 (119B-A6B)
AIME 2025	96.9	81.9	91.7	79.8
GPQA (Diamond)	84.3	68.1	78.0	74.7
LiveCodeBench v6	85.9	59.2	82.0	71.2
HumanEval+	92.7	76.8	94.7	92.8

Community Insights and Counterpoints

While the technical report was praised for its transparency, the release sparked discussion regarding the definition of "sovereignty."

"The sovereignty topic needs more attention in general... self-hosting the model is one piece of sovereignty, but how do we handle the rest of the agent stack — embeddings, retrieval, memory, etc.?"

Other critics pointed to the company's reported merger with Cohere, a Canadian firm, questioning if the "sovereign" claim remains valid if the company is operated outside of Europe. Additionally, some users noted that while the model is strong on benchmarks, real-world application in German bureaucracy still requires care, with one user reporting a failure to reference the most recent legislation (the Digital Services Act) over an outdated act.

Deployment

Kolibri is available on Hugging Face and requires the aleph-alpha-inference package for vLLM integration.

Recommended serving command:

vllm serve Aleph-Alpha/Kolibri-1 --kv-cache-dtype fp8 --reasoning-parser kolibri1 --tool-call-parser kolibri1 --enable-auto-tool-choice