AI & ML
impact 16
How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining arXiv:2511.18903v2 Announce Type: replace-cross Abstract: Due to the scarcity of high-quality data, large language models (LLMs) are ofte…
Why it matters
This signals a broader shift in data. The real question is whether learning moves the needle for practitioners.