typedef-ai/fenic

fenic – Semantic DataFrames for humans and agents

What it is

• A Python library that extends the familiar DataFrame/SQL style API (think pandas, PySpark, or Spark SQL) with semantic operators powered by large language models (LLMs).
• Operators such as extract, classify, summarize, embed, and even semantic joins are first‑class parts of the query plan, so LLM calls are managed automatically (batching, rate‑limiting, caching, cost accounting).

Why it matters

• Unstructured text (tickets, logs, transcripts, PDFs, etc.) is usually processed with ad‑hoc regexes or one‑off prompts, which are hard to reproduce.
• fenic turns that exploration into a typed, inspectable pipeline that can be versioned, rerun, and called by both humans and AI agents.
• The pipeline itself becomes the artifact: you can explain() the logical plan, trace row lineage, see token/cost metrics, and expose the result as a tool via the Model Context Protocol (MCP).

Core concepts

Concept	Description
Session	Holds configuration (app name, model providers, rate limits). All DataFrames are created from a session.
Semantic operators (fc.semantic.*)	Functions that invoke LLMs inside a query: extract(col, PydanticSchema), classify(col, classes), predicate(prompt, …), reduce(prompt, column), embed(col), etc.
Lazy pipeline	Operations are recorded as a logical plan and executed only when needed, allowing automatic batching and caching of LLM responses.
Typed output	Results are validated against Pydantic models, giving you columns with guaranteed types.
MCP integration	Tables/views can be published as typed tools that agents can call, turning the DataFrame result into a reusable AI tool.

Typical workflow (60‑second demo)

import fenic as fc
from pydantic import BaseModel, Field

class Ticket(BaseModel):
    product: str = Field(description="Product mentioned")
    sentiment: str = Field(description="positive, neutral, or negative")
    issue: str = Field(description="One‑line summary of the problem")

session = fc.Session.get_or_create(
    fc.SessionConfig(
        app_name="quickstart",
        semantic=fc.SemanticConfig(
            language_models={"mini": fc.OpenAILanguageModel(model_name="gpt-4o-mini", rpm=500)}
        ),
    )
)

df = session.create_dataframe([
    {"id": 1, "text": "The CSV export in Reports keeps timing out."},
    {"id": 2, "text": "Love the new dashboard, but SSO login is broken on mobile."},
])

tickets = (
    df.select("id", fc.semantic.extract("text", Ticket).alias("t"))
      .unnest("t")
)

tickets.show()

The LLM extracts the fields defined in Ticket and returns a regular DataFrame you can query, join, or export.

Key features

• Semantic joins – join two tables by describing the relationship in natural language.
• Embedding‑based joins (sim_join) and clustering (with_cluster_labels).
• Built‑in support for many source types – CSV, Parquet, Markdown, transcripts, PDFs, JSON, S3, Hugging Face datasets.
• Observability – explain(), lineage(), per‑query token/cost metrics, response cache.
• Extensible provider layer – OpenAI, Anthropic, Google, Cohere, OpenRouter; you can register multiple models and pick per‑operator.
• CLI helpers – fenic check (static linting of pipelines) and fenic skill install (installs a skill definition for coding agents like Claude Code, Cursor, etc.).

Example use cases

• Eval trace analysis – classify failure modes, aggregate root‑cause summaries, and publish the result as an MCP tool.
• Customer support – turn ticket text into structured rows for dashboards and downstream automation.
• Research agents – ingest news articles, podcasts, or meeting transcripts, extract entities, summarize, and store as queryable tables.
• Log triage – semantic extraction from free‑form logs, clustering of similar incidents, and generation of incident reports.

Installation

pip install fenic               # core
pip install "fenic[pdf,cluster,sim-join]"  # optional extras for PDF parsing, clustering, etc.

Set the appropriate provider API key (e.g., OPENAI_API_KEY).

Community & contribution

• Discord community for support.
• Example repository fenic-examples with notebooks covering many domains.
• Open contribution guidelines; PRs welcome.

Bottom line fenic is a genuine open‑source project that brings LLM‑powered semantic processing into a DataFrame abstraction, making unstructured data pipelines reproducible, inspectable, and shareable between humans and AI agents.