LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B

Overview

LLMON (LLM Object Notation) is a new markup language designed to explicitly communicate structure and semantics to LLMs, addressing the brittleness and security vulnerabilities inherent in unstructured or JSON-based context windows. IBM has released two experimental preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, which are trained to natively consume and generate LLMON-marked text.

Why LLMON Replaces JSON for LLMs

JSON was designed for machine parsers, not for the causal nature of LLMs. In JSON, structural delimiters like quotes and brackets are often indistinguishable from content, and transformers struggle with the long-range dependency tracking required for deep nesting.

LLMON uses an XML-like structure where delimiters are reserved special tokens rather than standard characters. This ensures that structure and content occupy separate parts of the model's vocabulary. By using unique IDs to bind instructions to specific data payloads, LLMON allows models to treat context as a series of addressable blocks rather than an opaque string of text.

Mitigation of Prompt Injection

LLMON significantly reduces the success rate of indirect prompt injection attacks by enforcing a strict separation between instructions and data. Because the model is trained to recognize specific execution blocks, it treats instructions embedded within data payloads as inert material rather than executable commands. Benchmarks on the OPI dataset show that LLMON models avoid 47% more attacks than the best-tested frontier models and 95% more than the best open-source models, all while maintaining general performance parity with standard Granite 4.1 Instruct models.

Integration and Training

LLMON is designed to complement existing chat templates rather than replace them. Developers can continue using standard messages lists, which are then rendered into LLMON blocks via the model's chat template. Each component—such as instructions, data, or requirements—can be marked as exec: True or False, effectively telling the model which parts of the context are authoritative.

The models were trained from Granite 4.1 base checkpoints using a multi-stage process:

  1. Re-serialization: Conventional instruction-tuning corpora were converted into LLMON-formatted blocks.
  2. Data Augmentation: The training set included decoy instructions and unbound data blocks to teach the model to ignore non-referenced information.
  3. Reinforcement Learning: The training involved five sequential GRPO stages covering reasoning, RLHF, and knowledge calibration, followed by a final DPO stage using preference pairs derived from the model's own responses to injection attempts.

Customization and Availability

LLMON is schema-agnostic, allowing users to define custom tag names and structures for specific domains. Users can fine-tune a LoRA adapter on top of the LLMON base models to understand domain-specific vocabularies, as demonstrated in the provided ScienceWorld examples. The project includes conversion tools to transform existing JSON into LLMON without data loss, and the models are available on Hugging Face for evaluation and experimentation.

Sources