morphik-org/morphik-core

Open-source multimodal retrieval engine (Morphik Core). By Morphik — AI back office for skilled nursing & senior living (morphik.ai).

What it solves

Morphik Core addresses the failure of traditional RAG (Retrieval-Augmented Generation) pipelines when dealing with visually rich documents. Standard text-based extraction often destroys the meaning of charts, diagrams, and tables, leading to AI applications that provide incorrect answers because they cannot "see" or understand the spatial relationships and visual data within a document.

How it works

It serves as a multimodal retrieval engine that allows developers to ingest, transform, and search unstructured data. Instead of relying on fragmented pipelines of OCR and text embeddings, it uses techniques like ColPali to enable search that understands visual content across PDFs, images, and videos through a single endpoint. It also provides tools for fast metadata extraction, including classification and bounding boxes.

Who it’s for

Developers building AI applications that need to integrate complex, nuanced context from multimodal documents without manually stitching together multiple extraction and retrieval tools.

Highlights

  • Multimodal Search: Uses ColPali to search across images, PDFs, and videos based on visual understanding.
  • Metadata Extraction: Capable of labeling, classification, and extracting bounding boxes from documents.
  • Flexible Access: Available via a Python SDK, REST API, a web-based Morphik Console, and the Model Context Protocol (MCP).
  • Broad Integrations: Connects with tools like Google Suite, Slack, and Confluence.