Think of it like this: how many books have been written? Millions. How many books are truly great? Not millions. Probably less than 10,000 depending on your definition of “great.” LLMs are trained on the full corpus, so most of what they learn from is not great. But they aren’t using the bad stuff to learn its substance. They are using it to learn patterns in human writing.