pixeltable/pixeltable

What it solves

Pixeltable simplifies the creation of multimodal AI applications by providing a unified backend. It eliminates the need to manually glue together separate components like blob storage, vector databases, orchestrators, and API endpoints, which typically requires complex ETL scripts and maintenance of separate glue scripts.

How it works

Pixeltable uses a Python API to manage multimodal data (images, video, audio, documents) and AI workflows. It employs "computed columns" that run incrementally on new or stale rows, allowing users to define AI processing pipelines (like embeddings or LLM summaries) directly in the table schema. It also provides built-in iterators to "explode" media into rows (e.g., splitting documents into chunks or videos into frames) and integrates a vector search index that stays in sync with the table data.

Who it’s for

Developers building multimodal AI data apps, ML engineers working on data wrangling for ML, and those creating RAG pipelines or AI agents that require structured and unstructured media storage and retrieval.

Highlights

• Unified Multimodal Interface: Store structured data and media (S3, GCS, Azure, etc.) in a single table.
• Detailed Iterators: Built-in tools to split documents, extract video frames, or segment audio.
• Declarative Orchestration: AI models and processing steps are defined as computed columns that handle caching and retries automatically.
• Integrated Vector Search: Combines semantic similarity search with metadata filtering in a single query.
• Automatic Serving: Generate REST API endpoints directly from the table schema via TOML configuration or FastAPI.
• Version Control: Built-in time travel capabilities to revert modifications or query specific snapshots of the table.
• Agentic Tooling: Support for tool calling and Model Context Protocol (MCP) for LLM-driven data operations.