MigoXLab/dingo
Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
What it solves
Dingo is a comprehensive evaluation tool designed to assess and improve the quality of AI data, models, and applications. It addresses the challenge of ensuring high-quality training data for LLMs, validating fine-tuning datasets, and detecting hallucinations in production RAG (Retrieval-Augmented Generation) systems.
How it works
It employs a hybrid evaluation strategy that combines fast, deterministic rule-based checks (heuristics and regex) with deep semantic analysis using LLMs and AI agents. The tool can connect to various data sources—including local files, SQL databases, HuggingFace datasets, and S3 storage—and process them using either a local executor for rapid iteration or a Spark executor for billion-scale datasets. It supports multi-field evaluation, allowing different quality rules to be applied to different fields in a single pass.
Who it’s for
ML practitioners, data engineers, and AI researchers who need to systematically validate their datasets and monitor the reliability of their AI systems in production.
Highlights
- Hybrid Evaluation: Combines 30+ built-in rules with LLM-based deep assessment and agent-based reasoning for multi-step fact verification.
- RAG Assessment: Provides five academic-backed metrics (Faithfulness, Answer Relevancy, Context Precision, Context Recall, and Context Relevancy) to evaluate retrieval and generation quality.
- Enterprise Data Integration: Supports memory-efficient streaming for massive SQL databases and direct integration with HuggingFace and S3.
- Extensible Architecture: Features a plugin system for registering custom rules, prompts, and models without modifying the core codebase.
- MCP Server: Includes a built-in Model Context Protocol server for integration with AI IDEs like Cursor and Claude Desktop.