zinggAI/zingg

What it solves

Zingg solves the problem of duplicate records across disparate data sources. In real-world data, the same entity (like a customer or patient) often appears multiple times with slight variations in their information, making it difficult to build a single source of truth for analytics and master data management.

How it works

Zingg uses machine learning to perform entity resolution. It employs two primary models to handle data at scale:

1. Blocking Model: To avoid comparing every single record with every other record (which would be quadratically expensive), Zingg learns a clustering model that indexes similar records together, drastically reducing the number of comparisons needed.
2. Similarity Model: A classifier that predicts whether a pair of records within the same block are matches, even if they are not identical.

To train these models, Zingg includes an interactive learner that uses active learning to build high-accuracy models from small training samples.

Who it’s for

This tool is designed for analytics engineers and data scientists who need to integrate data silos, perform large-scale deduplication, and establish unified views of business entities across multiple systems.

Highlights

• Active Learning: Interactive training data builder to create accurate models with minimal labeling effort.
• Scalability: Auto-learning blocking models allow the tool to scale to millions of records using Apache Spark.
• Broad Connectivity: Connects to various cloud data warehouses (Snowflake, S3, Azure), NoSQL databases, Cassandra, and major RDBMS.
• Multilingual Support: Out-of-the-box support for English, Chinese, Thai, Japanese, and Hindi.
• Flexible Entity Handling: Capable of handling any entity type, such as customers, patients, suppliers, or products.