apache/texera

Human-AI Collaborative Data Science Using Visual Workflows

Apache Texera – Visual, Human‑AI Collaborative Data‑Science Platform

What it is

  • An open‑source Apache Incubating project that lets analysts build, run, and tweak data‑analysis pipelines through a web‑based visual editor.
  • AI agents inside the system understand natural‑language instructions, so users can ask the platform to add operators, change parameters, or explore data without writing code.

Core capabilities

Feature What you get
Natural‑language data science Speak or type commands (e.g., “filter rows where age > 30”) and an AI agent translates them into workflow operators.
GUI‑driven workflows Drag‑and‑drop nodes (Python, Java, SQL, etc.) onto a canvas, connect them, and execute with a click.
Real‑time collaboration Multiple users can edit the same workflow simultaneously, seeing each other’s changes live.
Interactive debugging Inspect intermediate results, pause/resume execution, and modify the pipeline on the fly.
Language‑agnostic runtime Nodes can be written in Python or Java; the engine treats them uniformly.
Scalable parallel engine The backend distributes work across cores or a cluster, enabling big‑data processing from a laptop up to 100‑node, 400‑core deployments.
Separate compute & storage Storage can be swapped (e.g., local files, cloud object stores) without touching the compute layer, simplifying cloud deployments.

Typical use cases

  • Exploratory data analysis for scientists who want quick visual feedback while still being able to invoke sophisticated AI models.
  • AI‑for‑Science pipelines where domain experts guide machine‑learning steps (feature extraction, model training, result interpretation) via natural language.
  • Team analytics projects that require shared, versioned workflows and live collaboration.
  • Teaching data‑science concepts with an intuitive interface that hides boilerplate code.

Getting started

  1. Run locally – clone the repo and start the Docker‑compose stack (docker compose up). The web UI is then reachable at http://localhost:3000.
  2. Create a workflow – drag a Source node (CSV, database, etc.), connect it to processing nodes (filter, map, ML model), and finish with a Sink (file, dashboard).
  3. Ask the AI – open the chat‑like assistant panel, type a natural‑language command, and watch the system add or modify nodes automatically.
  4. Scale out – for larger jobs, deploy the backend on a cloud VM cluster; the system automatically distributes operators across available cores.

Community & impact

  • Users: 332 + registered users
  • Projects built: 86
  • Workflows executed: 2,481 (over 51 K individual executions, 357 K versioned runs)
  • Deployments: 7 public deployments, the biggest on 100 nodes / 400 cores
  • Publications: Described in a VLDB 2024 paper (citation provided).

Where to learn more

Apache Texera brings together visual workflow engineering and conversational AI, making sophisticated data‑science pipelines accessible to analysts who may not be expert programmers.

Related

  • Project
    vercel/next-evals-oss

    Next.js Evals is a Vercel‑maintained framework for automatically testing AI coding agents on real‑world Next.js tasks. It syncs eval fixtures from the `vercel/next.js` repo, runs agents in a Vercel sandbox, checks hidden assertions, records pass/fail, token usage, and cost, and exports a JSON leaderboard that powers the public Next.js evaluation site.

  • Project
    huanchong-99/SoloDawn

    SoloDawn is an open‑source web app that automates full‑stack software development. It orchestrates multiple AI command‑line tools (Claude Code, Codex, etc.) via a primary LLM‑driven agent that spawns child agents on demand. Tasks are executed in parallel Git branches, each passing three layers of quality checks. When the generated code meets a 90‑point rubric, it is auto‑merged, delivering a production‑ready product from a simple natural‑language description.

  • Project
    aa0101181514/tw-legal-rag

    tw‑legal‑rag is an open‑source Python CLI that connects to a hosted semantic‑search service containing 22 million+ Taiwanese judgments, statutes and administrative rulings. It lets you search with natural language, fetch exact case numbers, bundle results (including excerpts, case‑history and citation whitelists) into a JSON file, and run a deterministic citation‑check against LLM‑generated answers. The tool itself never calls an LLM and works without API keys, serving as a retrieval‑augmented generation (RAG) front‑end for legal AI applications.

  • Project
    EthanAlgoX/AIStock

    AI Stock is an open‑source, LLM‑driven research workspace for Chinese, Hong‑Kong and US equities. It lets a single trader start a natural‑language conversation, runs deterministic financial tools and optional expert‑agent collaborations (pipeline, debate, voting), and produces saved research reports and paper‑trade simulations. Built with FastAPI (Python) and a React 19 front‑end, it supports configurable LLM providers, strict capability whitelists, and a modular agent‑skill‑tool architecture. MIT‑licensed.