holoviz/lumen

What it solves

Lumen provides a way for users to chat with their data using natural language, allowing them to perform complex data analyses, generate visualizations, and build dashboards without needing to write code. It bridges the gap between raw data sources (like SQL databases or CSV files) and actionable insights by using LLMs to automate the data transformation and visualization process.

How it works

Lumen uses an agent-based framework and a declarative data model. This allows LLMs to generate entire data transformation pipelines (in SQL or Python) and visual outputs that can be serialized. Because it is built on Panel, it can render a wide variety of outputs. Users can provide context via documents and embeddings for retrieval augmented generation (RAG), and the resulting pipelines can be shared or integrated into notebooks and dashboards.

Who it’s for

It is designed for both programmers and non-programmers who need to extract insights from datasets, summarize results, and create interactive data visualizations from local files, remote files, or data lakes.

Highlights

• Natural Language Interface: Generate SQL, data pipelines, and charts using plain text.
• RAG Capabilities: Integration of documents and embeddings to provide LLMs with necessary context.
• Declarative Data Model: Enables easy serialization and sharing of analysis pipelines and dashboards.
• Extensible Architecture: Support for custom agents, tools, and domain-specific analyses.
• Inspectable Outputs: All LLM-generated results can be reviewed, validated, and manually edited.