The "biggest model that fits" instinct is just wrong now. Compact models routinely beat massive predecessors from 12 months ago. Scaling laws only reliably predict pre-training loss anyway, not how the model actually performs on your task. Dug into the research behind this: https://philippdubach.com/posts/the-most-expensive-assumptio...