dais-labo/flock

Beyond Quacking: Deep Integration of Language Models and RAG into DuckDB (VLDB Demo 2025)

What it solves

Flock is a DuckDB extension that allows users to perform semantic analysis and AI-driven data processing directly within a database. It bridges the gap between traditional structured data analytics (OLAP) and the capabilities of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) pipelines, enabling users to process unstructured data using standard SQL queries.

How it works

Flock integrates as a loadable extension for DuckDB, adding new SQL functions for AI tasks. It provides a declarative SQL interface to interact with various LLM providers (such as OpenAI, Azure, and Anthropic/Claude) and supports multimodal inputs including text, images, and audio (via transcription).

Who it’s for

Data analysts and developers who want to integrate LLM-based text generation, classification, and summarization into their existing SQL workflows without moving data out of the database environment.

Highlights

  • Declarative SQL Interface: Perform AI tasks like summarization and embedding generation using SQL.
  • Multi-Provider Support: Compatible with OpenAI, Azure, Ollama, Anthropic/Claude, and TypeSafe/Jev.
  • Multimodal Support: First-class support for text, images, and audio transcription.
  • Browser Support: Can be compiled as a DuckDB-WASM extension to run client-side in the browser.
  • LLM Observability: Built-in metrics tracking for token usage, latency, and execution time.

Related

  • Project
    Verified-Intelligence/auto_LiRPA

    A library for automatically computing provable output bounds for neural networks to ensure formal robustness and certified adversarial defense.

  • Project
    upgini/upgini

    An intelligent data search and enrichment library that automatically finds and integrates accuracy-improving external features into ML pipelines using LLMs and GNNs.

  • Project
    facebookresearch/recipes

    A library of ready-to-run deep learning training recipes that provide production-ready baselines for common tasks to eliminate repetitive glue code.

  • Project
    lovstudio/Ataru

    A high-performance local memory retrieval system that indexes AI session transcripts from various agents to make historical conversations searchable via GUI, CLI, or API.