For example, I'm currently studying probability theory at a fairly rigorous level (building up from measure theory) and my relative lack of skill with calculus is the main stumbling block. And I was recently talking with one of my professors who said that the biggest obstacle he sees students facing in trying to understand advanced mathematics isn't the difficulty of the math itself, but in their lack of mastery of the fundamental calculations that are necessary to follow the ideas and to come up with them on your own. Fluency with calculus is an enormous part of this. That said, I agree with the point, which is that for most students linear algebra is more useful.
Actually, I might go a bit farther and say that most students don't need to learn much math at all, at least in the way it is taught. Math is pretty poorly taught for a subject that is actually quite interesting once you start to understand it better (part of the problem is that in order to get to the interesting parts you need to first master all the boring stuff), and most people won't use any nontrivial math in their future lives.