techascent/tech.ml.dataset

What it solves

tech.ml.dataset (TMD) is a Clojure library designed for high-performance tabular data processing on the JVM. It provides a functional approach to handling large datasets, serving as a JVM-based alternative to tools like Python's Pandas or R's data.table.

How it works

TMD uses columnar storage and primitive arrays, packed datetime types, and string tables to reduce memory footprint and increase processing speed. Unlike traditional data frames in Python or R, TMD datasets are functional, making them easier to reason about during data manipulation.

Who it’s for

Data scientists and developers using Clojure or Java who need to process data-intensive workloads efficiently on the JVM.

Highlights

• Columnar Storage: Reduces memory usage through efficient data layout.
• Functional API: Provides a predictable, functional interface for data manipulation.
• JVM Integration: Fully compatible with Clojure and provides a Java API for integration into Java projects.
• High Performance: Optimized for speed and memory efficiency using primitive arrays.