NVIDIA-NeMo/Guardrails

What it solves

LLM-powered chatbots and assistants can go off-topic, leak sensitive data, hallucinate facts, or be manipulated by jailbreak and prompt-injection attacks. NeMo Guardrails is a toolkit that lets developers insert a programmable safety and control layer between their application and the LLM, so they can define rules for what the model may say, which topics it should avoid, how it should respond in certain situations, and which tools it can call safely.

How it works

Instead of calling the LLM directly, the application calls the guardrails layer using methods like generate(). The layer runs a series of configurable “rails” that process the user input, guide how the LLM is prompted, check retrieved documents (for RAG), validate tool calls, and inspect the model's output before it reaches the user. Guardrails are defined in a configuration folder containing a config.yml file (LLM models, active rails, settings), optional actions.py for custom Python actions, and .co files written in Colang, a dialogue-modeling language. Colang lets you specify conversational flows and rules in a readable, Python-like syntax. The library supports five rail types: input rails, dialog rails, retrieval rails, execution rails, and output rails. It also includes a CLI to run a guardrails server, a Docker image, optional LangChain integration, and built-in guardrails for jailbreak detection, content safety, fact-checking, and hallucination detection.

Who it’s for

Developers building LLM-based conversational applications who need more control and safety than a raw model prompt provides. It is aimed at teams creating question-answering bots, domain-specific assistants, or exposed LLM endpoints, and who want to enforce compliance, prevent harmful content, and design predictable conversation flows. Users should be comfortable with Python and basic YAML configuration; knowledge of Colang is helpful but the syntax is designed to be easy to learn.

Highlights

• Five types of programmable guardrails: input, dialog, retrieval, execution, and output.
• Colang dialogue modeling language with a Python-like syntax for defining conversational flows and rail conditions.
• Built-in guardrails for jailbreak and prompt-injection detection, content safety, fact-checking, and hallucination detection.
• Works with multiple LLMs (OpenAI GPT-3.5/GPT-4, LLaMa-2, Falcon, Vicuna, Mosaic) and integrates with LangChain optionally.
• Includes CLI, server mode with an HTTP API, and Docker support for deployment.
• Provides an evaluation tool (nemoguardrails evaluate) to test conversational safety.
• Active open-source project by NVIDIA with extensive documentation and examples.