NVIDIA/NeMo-Retriever

NeMo Retriever Library is a scalable, performance-oriented document content and metadata extraction microservice. NeMo Retriever Library uses specialized NVIDIA NIM microservices to find, contextualize, and extract text, tables, charts and images that you can use in downstream generative applications.

What it solves

NeMo Retriever solves the challenge of extracting structured content from complex documents—such as PDFs containing text, tables, charts, and infographics—and preparing that data for use in retrieval-augmented generation (RAG) and other generative AI applications.

How it works

The framework implements a scalable pipeline that splits documents into pages and classifies artifacts. It uses optical character recognition (OCR) and various models to extract content into a defined JSON schema. Once extracted, the library computes embeddings for the content and stores them in LanceDB for efficient retrieval. The pipeline can be run in batch mode for small workloads or deployed via Helm on Kubernetes for production-level scalability.

Who it’s for

It is designed for developers and engineers building enterprise RAG applications who need to ingest multimodal document data (text and visuals) at scale.

Highlights

  • Multimodal Extraction: Specifically handles text, tables, charts, and infographics.
  • Scalable Deployment: Supports Kubernetes and Helm for production environments.
  • Flexible Integration: Compatible with NVIDIA NIM microservices, HuggingFace models, and retrieval pipelines like LlamaIndex and LangChain.
  • Structured Output: Converts extracted document elements into a well-defined JSON schema or Markdown format.

Related

  • Project
    NVIDIA-AI-Blueprints/rag

    A reference solution for building enterprise RAG pipelines using NVIDIA NIM microservices, supporting multimodal data extraction and agentic multi-hop retrieval.

  • Project
    HKUDS/RAG-Anything

    An all-in-one multimodal RAG framework that enables seamless processing and querying of documents containing interleaved text, images, tables, and mathematical equations.

  • Project
    morphik-org/morphik-core

    A multimodal retrieval engine that enables AI applications to search and understand visually rich documents, such as PDFs and charts, without losing spatial or visual context.

  • Project
    aryn-ai/sycamore

    An AI-powered document processing engine for ETL and RAG that transforms unstructured documents into high-quality data chunks for vector databases.

  • Project
    OpenBMB/VisRAG

    A vision-language RAG pipeline that embeds documents as images to prevent information loss from text parsing and uses evidence-guided reasoning for multi-image QA.