NVIDIA and IIT Madras Drop 2.8T-Token Indic Corpus

An open pipeline for 22 Indian languages gives model builders the data—and the recipe—to build on.

By The ai-compute Desk

NVIDIA and IIT Madras' AI4Bharat lab have released Nemotron-pretraining-Indic-v1, a corpus of approximately 2.8 trillion tokens spanning all 22 languages listed in India's Constitution—the first languages for 97% of Indians, or nearly 1.2 billion citizens at the 2011 Census. [1]

[1]

Only outputs scoring at least 75 out of 100 are retained; feedback-based rewriting recovered 438.4 billion tokens already counted in the corpus total. [1]

The dataset is already proving its weight. Continued pretraining for Nemotron 3.5 Super VL added 10 billion Indic tokens, drawn from a twelve-language subset of this corpus and natural Indic text, producing gains of 4.98 to 9.82 percentage points across five Indic knowledge and reasoning benchmarks. [1]

The release lands at a moment when the gap between high-resource and low-resource language models has become hard to ignore. Opening both the data and the generation recipe shifts the dynamic: builders no longer have to start from scratch.

The full generation pipeline—prompts, quality checks, and repair workflows—is open-source, so teams can adapt it to their own source documents or domains. AI4Bharat is also releasing Indic-LLM-Eval-Suite, a benchmark collection designed to assess both task performance and language quality in Indian languages. [1]

we do not have enough data for Indian languages.Mitesh Khapra