typedef-ai/fenic

Semantic DataFrames for humans and agents

What it solves

fenic is designed to solve the brittleness of working with unstructured data. Traditionally, extracting meaning from logs, transcripts, and documents requires a mix of fragile regex scripts, one-off prompts, and manual notebook exploration. fenic transforms these ephemeral discoveries into durable, typed, and reusable DataFrame pipelines that can be shared between humans and AI agents.

How it works

It functions as a semantic DataFrame engine that integrates LLM operations directly into the query model. Instead of manually orchestrating API calls, users write PySpark/SQL-style operations (select, filter, join) alongside semantic operators like extract, classify, and summarize.

Key technical mechanisms include:

  • Lazy Execution: Pipelines are built lazily and compiled for inference, featuring automatic batching, rate limiting, and response caching.
  • Typed Schemas: Users define the desired output shape using Pydantic models, ensuring unstructured text is converted into structured, queryable columns.
  • Semantic Joins: The engine can join two DataFrames based on natural-language predicates (meaning) rather than exact key matches.
  • Lineage Tracking: It provides row-level lineage to trace outputs back to their source data.
  • MCP Integration: Pipelines can be promoted to Model Context Protocol (MCP) tools, allowing AI agents to call them as governed tool surfaces.

Who it’s for

  • Data Engineers and Analysts who need to process large volumes of unstructured text into structured datasets.
  • AI Developers building evaluation pipelines to triage model failures and identify patterns in traces.
  • Agent Developers who want to turn their agent's data exploration discoveries into repeatable, governed tools.

Highlights

  • Built-in Semantic Operators: First-class support for extraction, classification, sentiment analysis, and summarization.
  • Native Unstructured Support: Specialized handling for Markdown, transcripts (SRT/WebVTT), JSON (via jq), and PDFs.
  • Inference Optimization: Integrated token/cost accounting, retries, and caching to manage LLM overhead.
  • AI Agent Tooling: Includes fenic check for linting and fenic skill install to teach coding agents how to use the API correctly.
  • Multi-Provider Support: Compatible with OpenAI, Anthropic, Google, Cohere, and OpenRouter.