Hugging Face Blog·· 12 hours agoFeaturedAI Score60
NVIDIA and AI4Bharat release Nemotron-pretraining-Indic-v1, a 2.8T-token Indic corpus with an open data generation pipeline
Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline
AI summary
NVIDIA, with IIT Madras and its AI4Bharat lab, introduced Nemotron-pretraining-Indic-v1.
Why it matters
The release pairs a 2.8T-token Indic corpus with the open generation recipe, so teams can reuse the pipeline rather than only the data.
Source: Hugging Face Blog · huggingface.co