Llama Guard 4 and Llama Prompt Guard 2 Release
Meta has released Llama Guard 4, a 12B multimodal safety model, and Llama Prompt Guard 2, a series of classifiers designed to detect prompt injections and jailbreaks. These tools enable developers to filter both user inputs and model outputs to prevent the generation of unsafe content in production environments.
Llama Guard 4: Multimodal Safety Moderation
Llama Guard 4 is a dense 12B parameter model designed to detect inappropriate content across both text and images. It can be deployed on a single GPU with 24 GB of VRAM, making it a practical choice for production moderation pipelines. The model is capable of evaluating text-only and image+text inputs, allowing it to filter prompts before they reach a primary LLM and review assistant responses before they are delivered to the user.
Technical Architecture and Training
Llama Guard 4 utilizes a dense feedforward early-fusion architecture. It was created by pruning the Llama 4 Scout model, which uses a Mixture-of-Experts (MoE) architecture. By removing all routed experts and router layers and retaining only the shared expert, Meta developed a dense model initialized from pre-trained shared expert weights without requiring additional pre-training.
Training data for Llama Guard 4 consists of a 3:1 ratio of text-only to multimodal data, incorporating multi-image training data (up to 5 images) and human-annotated multilingual data previously used for Llama Guard 3.
Hazard Classification and Customization
Llama Guard 4 classifies content based on 14 hazard types defined by the MLCommons hazard taxonomy, plus code interpreter abuse. The supported categories include:
- S1: Violent Crimes
- S2: Non-Violent Crimes
- S3: Sex-Related Crimes
- S4: Child Sexual Exploitation
- S5: Defamation
- S6: Specialized Advice
- S7: Privacy
- S8: Intellectual Property
- S9: Indiscriminate Weapons
- S10: Hate
- S11: Suicide & Self-Harm
- S12: Sexual Content
- S13: Elections
- S14: Code Interpreter Abuse (text only)
Users can configure the list of detected categories during inference by excluding specific category keys (e.g., excluded_category_keys=["S9", "S2", "S1"]) via the chat template.
Performance Benchmarks
Compared to Llama Guard 3, Llama Guard 4 shows improvements in several areas, particularly in multi-image and English text performance:
| Category | Recall | False Positive Rate | F1-score | Δ Recall | Δ False Positive Rate | Δ F1-Score |
|---|---|---|---|---|---|---|
| English | 69% | 11% | 61% | +4% | -3% | +8% |
| Multilingual | 43% | 3% | 51% | -2% | -1% | 0% |
| Single-image | 41% | 9% | 38% | +10% | 0% | +8% |
| Multi-image | 61% | 9% | 52% | +20% | -1% | +17% |
Llama Prompt Guard 2: Injection and Jailbreak Detection
Llama Prompt Guard 2 consists of two new classifiers with 86M and 22M parameters. These models are focused specifically on detecting prompt injections and jailbreaks.
Key improvements over Llama Prompt Guard 1 include:
- Enhanced Performance: Improved detection capabilities.
- Efficiency: A new, faster and more compact 22M parameter model.
- Resilience: Tokenization that is more resistant to adversarial attacks.
- Simplified Classification: A binary classification system (benign vs. malicious).
Implementation and Deployment
To use these models, users must install hf_xet and a preview release of the transformers library (v4.51.3-LlamaGuard-preview).
Llama Guard 4 can be implemented using Llama4ForConditionalGeneration and AutoProcessor, while Llama Prompt Guard 2 can be deployed via the Hugging Face pipeline API or the AutoModelForSequenceClassification API.