Nemotron-Personas-India: Synthesized Data for Sovereign AI

NVIDIA has released Nemotron-Personas-India, a large-scale synthetic dataset of Indic personas designed to provide a privacy-preserving foundation for building Sovereign AI systems tailored to India's demographic, geographic, and cultural distributions. This release addresses the data gap where most open datasets reflect Western norms, enabling the development of AI that is more representative of India's multilingual and multi-script environment.

Dataset Composition and Scale

Nemotron-Personas-India consists of 21 million personas (derived from 3 million records, each containing 7 personas). The dataset is licensed under CC BY 4.0 and provides a comprehensive set of attributes to ground AI models in Indian reality.

Key Statistics

  • Total Tokens: 7.7 billion tokens, including 2.9 billion persona tokens.
    • English: 1 billion total tokens (394 million persona tokens).
    • Hindi (Devanagari): 4.7 billion total tokens (1.8 billion persona tokens).
    • Hindi (Latin): 2 billion total tokens (746 million persona tokens).
  • Demographic Coverage: Represents all 36 Indian states and 640 districts.
  • Diversity Metrics: Includes approximately 560,000 unique full names and 2,900 occupational categories, covering formal, informal, and traditional sectors (e.g., farming, street vending).
  • Data Fields: Each record contains 27 fields, including age, gender, education, occupation, state, district, and natural language fields for cultural background, skills, hobbies, and linguistic background.

Technical Implementation and Pipeline

The dataset was produced using NeMo Data Designer, NVIDIA's synthetic data generation microservice. This compound AI system utilizes Jinja templating, Pydantic validation, and structured outputs to scale generation.

Generation Models

Two primary models were used in the pipeline:

  1. Probabilistic Graphical Model (Apache-2.0): Used for statistical grounding to ensure the data reflects real-world distributions.
  2. GPT-OSS-120B (Apache-2.0): Used for narrative generation across English, Hindi (Devanagari), and Hindi (Latin).

Cultural and Statistical Grounding

To ensure authenticity, the dataset is aligned with official demographic distributions from the 2011 Census and Parsed Indian Electoral Rolls. It specifically incorporates:

  • Expanded Education and Occupation: Inclusion of diverse academic pathways and traditional sectors.
  • Life Stages: Categories for students, homemakers, and the retired/unemployed.
  • Digital Divide: Modeling of usage patterns based on income, age, and urban/rural divides.
  • Linguistic Diversity: Detailed mapping of first, second, and third spoken languages for each persona.

Privacy and Safety

Nemotron-Personas-India is private by design. Because all personas are fully synthetic, there is no risk of re-identification or ties to any living or deceased individuals. This allows developers to train models without the regulatory barriers associated with sensitive personal data.

Practical Applications for AI Development

This dataset enables developers to build region-aware AI agents and domain-specific copilots that generalize across India's diverse communities. Key applications include:

  • Multilingual Fine-tuning: Generating training data in Indian languages and scripts to handle complex multi-turn conversations.
  • Cultural Nuance: Fine-tuning models to capture local social and occupational nuances.
  • **Sovereign AI: Supporting the development of AI systems that incorporate region-specific demographics to mitigate bias and prevent model collapse caused by uncurated synthetic data.

Integration and Availability

The dataset is available via the Hugging Face datasets library. Users can load specific subsets based on language and script:

from datasets import load_dataset

# English personas
nemotron_personas_en = load_dataset("nvidia/Nemotron-Personas-India", "en_IN")
# Hindi personas in Devanagari
nemotron_personas_hi_deva = load_dataset("nvidia/Nemotron-Personas-India", "hi_Deva_IN")
# Hindi personas in Hindi (Latin)
nemotron_personas_hi_latn = load_dataset("nvidia/Nemotron-Personas-India", "hi_Latn_IN")

An extended version containing synthetic addresses, religion, and first/last names is available through NeMo Data Designer.

Sources