InternScience/GraphGen

What it solves

GraphGen is a framework designed to improve the Supervised Fine-Tuning (SFT) of Large Language Models (LLMs) by generating high-quality, knowledge-driven synthetic data. It specifically targets the problem of "knowledge gaps" in LLMs—areas where the model lacks information or is poorly calibrated—by prioritizing the generation of QA pairs for high-value, long-tail knowledge.

How it works

GraphGen follows a structured pipeline to create synthetic training data:

1. Knowledge Graph Construction: It builds a fine-grained knowledge graph from source texts (PDFs, JSON, TXT, etc.) or external databases (UniProt, NCBI, Wikipedia).
2. Gap Identification: It uses the expected calibration error metric to identify where an LLM is lacking knowledge.
3. Targeted Generation: It generates QA pairs based on the knowledge graph, using multi-hop neighborhood sampling to capture complex relations and style-controlled generation for diversity.
4. Data Augmentation: It includes a rephrase pipeline that uses LLM-driven reformulation to create diverse variants of the same corpus to avoid redundancy during pretraining.

Who it’s for

This tool is intended for AI researchers and developers who need to synthesize high-quality training data for LLMs, particularly in knowledge-intensive domains like medicine, law, or biology (e.g., plant science).

Highlights

• Diverse Data Types: Supports the generation of Atomic, Aggregated, Chain-of-Thought (CoT), Multi-hop, VQA (Visual Question Answering), and benchmark-style data (multiple-choice, fill-in-the-blank, true/false).
• Extensive Integration: Compatible with various inference backends (vLLM, SGLang, HuggingFace) and API servers (OpenAI, Azure, Ollama).
• Flexible Data Sources: Can ingest data from files, search engines (Google, Bing), and specialized bioinformatics databases.
• Scalable Architecture: Utilizes Ray for distributed execution and supports KuzuDB and RocksDB for efficient storage.