morphik-org/morphik-core

Open-source multimodal retrieval engine (Morphik Core)

What it solves

Morphik Core addresses the failure of traditional RAG (Retrieval-Augmented Generation) pipelines when dealing with visually rich documents. Standard text-based extraction often destroys the meaning of charts, diagrams, and tables, leading to AI applications that provide incorrect answers because they cannot "see" or understand the spatial relationships and visual data within a document.

How it works

It serves as a multimodal retrieval engine that allows developers to ingest, transform, and search unstructured data. Instead of relying on fragmented pipelines of OCR and text embeddings, it uses techniques like ColPali to enable search that understands visual content across PDFs, images, and videos through a single endpoint. It also provides tools for fast metadata extraction, including classification and bounding boxes.

Who it’s for

Developers building AI applications that need to integrate complex, nuanced context from multimodal documents without manually stitching together multiple extraction and retrieval tools.

Highlights

  • Multimodal Search: Uses ColPali to search across images, PDFs, and videos based on visual understanding.
  • Metadata Extraction: Capable of labeling, classification, and extracting bounding boxes from documents.
  • Flexible Access: Available via a Python SDK, REST API, a web-based Morphik Console, and the Model Context Protocol (MCP).
  • Broad Integrations: Connects with tools like Google Suite, Slack, and Confluence.

Related

  • Project
    tonywu71/colpali-cookbooks

    A collection of notebooks for implementing, fine-tuning, and interpreting ColPali and ColQwen2 models for vision-based document retrieval.

  • Project
    miurla/morphic

    An AI-powered search engine that uses a generative UI to render rich, interactive components and cited answers instead of plain text.

  • Project
    docling-project/docling

    Docling is an open‑source Python library/CLI that parses many document types (PDF, Word, slides, spreadsheets, HTML, e‑books, audio, video, email, etc.) into a unified, structured representation. It extracts layout, tables, code, formulas, images, and speech‑to‑text, then exports to Markdown, HTML, JSON, DocLang, DocTags, etc. Designed for generative‑AI workflows, it offers plug‑and‑play integrations with LangChain, LlamaIndex, Crew AI, Haystack, and supports local execution for privacy‑sensitive data.

  • Project
    ggozad/haiku.rag

    haiku.rag is a Python library/CLI for building a private, citation‑rich RAG system over PDFs, Office files, HTML, Markdown and images. It uses LanceDB for hybrid vector/full‑text indexing, Docling for document parsing, and Pydantic AI for LLM/embedding providers (defaulting to Ollama). Features include hybrid search, optional reranking, multimodal image search, sandboxed Python for data aggregation, and a CLI/TUI that shows exact page‑level citations. It offers an optional background ingester service and an MCP server for integration with tools like Claude Code. Install via `pip install haiku.rag` (or the slim core) and interact via commands like `haiku-rag add-src`, `haiku-rag ask`, or the async Python API.

  • Project
    Osilly/Vision-DeepResearch

    A framework and benchmark for developing Multimodal Large Language Models capable of deep research through iterative visual and textual search across images and videos.