The challenge is that when you get into scaling, you need to begin engineering for throughput, which requires a completely different engineering style focused on higher throughput and reduced variance. This style is well supported by "test absolutely everything" TDD.
Oh, and then when you want to do a tangential experiment, you want to go back to latency-oriented engineering, but without destabilizing existing code.
In short, I'd say yes, it's plausible that overuse of TDD could be hampering a startups agility.