mostly-ai/mostlyai

What it solves

It addresses the challenge of accessing high-fidelity data for analysis and testing while maintaining privacy. It allows users to create synthetic versions of tabular or language datasets that preserve the statistical properties of the original data without exposing sensitive information.

How it works

The SDK provides a programmatic interface to train "Generators" on existing data assets. These generators use various models—such as TabularARGN for tabular data, DNNs for graph relations, and fine-tuned Hugging Face models or LSTMs for text—to learn the underlying patterns of the data. Once trained, these generators can be used to produce synthetic datasets, probe for specific representative samples, or perform conditional simulations based on fixed column values.

Who it’s for

It is designed for data scientists and developers who need privacy-safe datasets for machine learning, software testing, or data sharing, and who want the flexibility to run these processes either on their own local compute or via a remote endpoint.

Highlights

• Broad Data Support: Handles mixed-type data (categorical, numerical, geospatial, text), multi-table structures, and time-series.
• Flexible Deployment: Offers a LOCAL mode for on-premise compute and a CLIENT mode for remote endpoints.
• Advanced Synthesis: Supports up-sampling, conditional simulations, re-balancing underrepresented segments, and statistical fairness controls.
• Quality Assurance: Includes automated quality metrics for fidelity and privacy, with detailed HTML reports for visual analysis.
• Integration: Built-in connectors for various databases and cloud storage (e.g., Snowflake, Postgres, BigQuery).