So despite it making no financial sense on an individual, house-by-house level, what happened in practice is that the market itself got strangled by this collusion and despite the existence of many empty houses available for either purchase or rent, you ended up with a non-negligible amount of these places not ending up in the market organically and as such driving up the costs for everyone. The property owners win big with this scheme of course, and the way things were setup you'd be a fool to go against the grain and try outcompete on cheaper housing, since you can't beat an entire market getting strangled.
So yes, "the rich" aren't that stupid, they just so happen to have systems that make these sort of moves profitable.