That's true, and I want to extend that by saying that at the highest levels life is more about relationships and who can do favors for you. Even Y Combinator is an example of this. I'm sure everyone who gets in is fully capable, but there are probably just as many capable people who don't get in. Those who got in will probably go on to do bigger things simply by having the Y Combinator brand and network. I often ask my friends at business school whether they feel it's worth it. Invariably they mention how the things they learn aren't that useful, but that the relationships they make (plus the signal of HBS or Wharton or wherever) are well worth the price of admission.

The downside of all this is that it promotes disparate outcomes for those who come from privileged backgrounds and those who come from more ordinary ones. I'm currently attending an "elite" college and there seems to be a pretty clear distinction between job placement and family status.