Near-Deduplication Strategies for Large Text Corpora
Redundancy in training data distorts scaling laws and wastes compute.
Freya Reyes
Correspondent
Freya Reyes is a correspondent at The Data Bench covering text & nlp data. Based in San Francisco, Freya has written for The Data Bench since 2020.
1 story · San Francisco
Redundancy in training data distorts scaling laws and wastes compute.