dflib/dflib

In-memory Java DataFrame library

What it solves

Java and JVM developers often need to manipulate tabular data in memory, similar to what pandas or R DataFrames provide. SQL is declarative and less composable for step-by-step transformations. DFLib gives you a DataFrame structure with operations like joins, filtering, and aggregation, applied programmatically.

How it works

DFLib implements the DataFrame concept as a pure Java library. You work with dynamic tables, applying operations step by step. It includes connectors to read and write common formats: CSV, Excel, databases, Avro, Parquet, JSON, and can be extended. It also integrates with Jupyter Notebook via a Java kernel, and can generate interactive charts using Apache ECharts (HTML/JavaScript output).

Who it’s for

Java and JVM developers building applications that need in-memory data processing, ETL tasks, log analysis, spreadsheet handling, or data exploration. It’s also suitable for data scientists working in Jupyter with Java.

Highlights

  • Pure Java implementation, no external dependencies.
  • SQL-like operations (joins, etc.) applied in-memory and composable.
  • Connectors for CSV, Excel, RDBMS, Avro, Parquet, JSON.
  • Integration with Jupyter Notebook via a community-maintained Java kernel.
  • Built-in chart generation with Apache ECharts for HTML/JavaScript visualization.

Related

  • Project
    edoliberty/long-term-memory-in-ai

    This repository provides course materials for a graduate class on vector search and databases, taught by experts from Pinecone and Meta. It covers embeddings, approximate nearest neighbor search, indexing techniques, and practical systems like FAISS, with lecture notes, a syllabus, and project options.

  • Project
    beowolx/rensa

    A high-performance MinHash library written in Rust with Python bindings for fast near-duplicate document detection and Jaccard similarity estimation.

  • Project
    fidetolabs/qanat

    An MCP server for building and backtesting financial valuation indicators with strict point-in-time data handling and trial tracking to prevent over-fitting.

  • Project
    Azure-Samples/azure-search-power-skills

    A collection of sample implementations and templates for creating custom Web API enrichment skills to extend the indexing pipelines of Azure AI Search.