tabularis-ai/be_great

What it solves

GReaT (Generation of Realistic Tabular data) is a framework designed to create high-quality synthetic tabular data. It addresses the problem of needing realistic data for testing, privacy-preserving data sharing, and augmenting small datasets without compromising the original sensitive information.

How it works

The framework leverages pretrained Transformer-based language models (LLMs) to treat tabular data as a sequence of tokens. It can operate in three primary modes:

1. Training-based Generation: The model is fine-tuned on a real dataset using fit() and then generates new samples via sample().
2. Mock Data Generation: It can generate synthetic rows from a declarative schema (defining types, ranges, and distributions) without requiring any real training data.
3. Imputation: It can fill in missing values (NaNs) in existing datasets based on the learned distribution.

To optimize performance, it supports LoRA (Low-Rank Adaptation) for efficient fine-tuning and provides "guided sampling" for complex feature sets to ensure higher quality generation.

Who it’s for

Data scientists, ML engineers, and developers who need synthetic tabular data for privacy-safe dummy data, test fixtures, dev environments, and augmenting datasets for downstream ML tasks.

Highlights

• Conditional Generation: Generate data that satisfies specific logical constraints (e.g., age >= 30).
• Live Quality Monitoring: Track synthesis quality using ColumnShapes similarity during the training process.
• Comprehensive Evaluation Suite: Built-in metrics for statistical similarity, downstream utility (ML Efficiency), and privacy (e.g., k-Anonymization, Membership Inference).
• Efficient Fine-Tuning: Integration with LoRA to reduce memory and training time on consumer hardware.
• Flexible Schema-based Mocking: Create realistic data from JSON/YAML schemas without needing a training set.