First, let's qualify that statement. Is music, like mathematics, a young person's game?
Musicians tend to release their classic albums, i.e. what general consensus agrees is their best work, in their early years, not when they're 40+. Look at the biggest, long-running music acts today, such as The Rolling Stones. No one is going to their concerts to hear their latest material. They want to hear Jumpin' Jack Flash (released ~1968).
The early deaths of legends like Jimi Hendrix, Jim Morrison, Kurt Cobain etc. are tragic, sure, but does anyone really believe these guys would have put out their best work at 40? The people lamenting wasted potential are perhaps misunderstanding how musical creativity works.
In contrast, writers tend to put out their best work after their formative years. If you think of your favorite writer, I'm willing to bet their magnum opus wasn't released before 30.
Why is this? What accounts for the difference?