Preprint Open access
Adapting English Quality Classifiers for Multilingual LLM Pretraining Data Selection
Recent advances in large language model (LLM) pretraining highlight the role of high-quality training data in improving performance. While model-based filtering has proven effective in selecting high-quality subsets from web-scale corpora, especially for high-resource languages, low-resource languages face challenges d …