NVIDIA/NeMo-Retriever

NeMo Retriever Library is a scalable, performance-oriented document content and metadata extraction microservice. NeMo Retriever Library uses specialized NVIDIA NIM microservices to find, contextualize, and extract text, tables, charts and images that you can use in downstream generative applications.

What it solves

NeMo Retriever Library (also known as NVIDIA Ingest) solves the challenge of preparing complex, unstructured documents—such as PDFs containing text, tables, charts, and infographics—for use in Retrieval-Augmented Generation (RAG) applications. It automates the extraction, classification, and conversion of these document elements into a structured format suitable for AI models.

How it works

The library provides a scalable pipeline that processes documents by splitting them into pages and classifying content types. It uses OCR and specialized extraction techniques to convert these elements into a standardized JSON schema. The pipeline then computes embeddings for the extracted content and stores them in a vector database (LanceDB). The process is designed to be lazy and chainable, allowing developers to define ingestion tasks that are executed in batch or in-process modes.

Who it’s for

This project is for developers and data engineers building enterprise-grade RAG pipelines who need to ingest diverse document types at scale. It is particularly useful for those already working within the NVIDIA AI ecosystem, as it integrates with NVIDIA NIM microservices and supports deployment on Kubernetes via Helm for production workloads.

Highlights

  • Supports multi-modal extraction including text, tables, charts, and infographics.
  • Provides a chainable, lazy-loading Python API for building custom ingestion pipelines.
  • Enables scalable production deployment using Kubernetes and Helm charts.
  • Integrates seamlessly with vector databases like LanceDB and popular frameworks like LangChain and LlamaIndex.

Related

  • Project
    NVIDIA-NeMo/Megatron-Bridge

    NVIDIA‑NeMo Megatron‑Bridge is a PyTorch library that links Hugging Face models with NVIDIA’s Megatron‑Core training engine. It enables bidirectional checkpoint conversion, large‑scale pre‑training and fine‑tuning (including LoRA), and provides verified recipes for many recent LLM/VLM/MoE models. The recommended workflow uses NVIDIA’s NeMo Docker container and a simple Python API (`AutoBridge`).

  • Dispatch
    NeoMME: Efficient Multimodal-native and Multilingual Encoder

    Hugging Face introduces NeoMME, a family of multimodal encoders (260M and 800M parameters) that use a single bidirectional Transformer to process text and images from scratch, optimizing visual document retrieval.

  • Project
    NVIDIA-NeMo/Gym

    NeMo Gym is NVIDIA’s open‑source library for scalable evaluation and training of AI agents in stateful environments (code execution, tool calling, safety, etc.). It provides modular interfaces, a hub of ready‑made benchmarks, support for thousands of concurrent runs, and integration with any RL framework. Install via `uv`, configure a model (e.g., OpenAI), start the environment servers, and run the `gym eval` CLI to collect rollouts and metrics.

  • Dispatch
    Domain-Specific Embedding Fine-Tuning with NVIDIA Nemotron – Under a Day

    NVIDIA and Hugging Face released a single‑GPU, under‑a‑day pipeline that fine‑tunes the Llama‑Nemotron‑Embed‑1B‑v2 model on synthetic domain data, delivering >10% retrieval gains and up to 26% improvement on real enterprise datasets.

  • Project
    NVIDIA-NeMo/Guardrails

    NVIDIA NeMo Guardrails is an open-source toolkit that adds programmable safety and control layers to LLM-based conversational apps. It uses configurable rails and a Python-like dialogue language (Colang) to prevent unwanted topics, detect jailbreaks and injections, fact-check outputs, and manage tool calls, all while integrating with popular LLMs and LangChain.