NVIDIA-NeMo/Guardrails

What it solves

It provides a programmable layer between an application and a Large Language Model (LLM) to ensure conversations remain safe, secure, and on-topic. It prevents LLMs from discussing unwanted topics, protects against vulnerabilities like jailbreaks and prompt injections, and ensures the model follows predefined conversational paths rather than deviating into unpredictable behavior.

How it works

The library implements "rails"—programmable controls that intercept and modify data at different stages of the LLM pipeline:

• Input Rails: Screen or alter user input before it reaches the LLM.
• Dialog Rails: Use a specialized modeling language called Colang to define canonical forms and steer the conversation flow.
• Retrieval Rails: Filter or mask data retrieved during RAG (Retrieval Augmented Generation) processes.
• Execution Rails: Control the inputs and outputs of custom tools or actions called by the LLM.
• Output Rails: Validate or modify the LLM's response before it is delivered to the user.

Who it’s for

Developers building LLM-powered applications, such as domain-specific chatbots, RAG-based question-answering systems, or custom LLM endpoints, who need strict control over model behavior and safety.

Highlights

• Colang Modeling Language: A Python-like language designed specifically for designing controllable dialogue flows.
• Comprehensive Protection: Built-in support for hallucination detection, fact-checking, and jailbreak/injection scanning.
• Flexible Integration: Works with various LLMs (GPT-4, LLaMa-2, etc.) and integrates optionally with LangChain.
• Deployment Options: Can be used as a Python library, via a CLI, or deployed as a standalone server with an HTTP API.