That's where vacancy taxes should be kicking in. People need to get it through their heads: you don't have some god-given right to hoard property for any other purpose than for it to be legit occupied. It needs to be discouraged as an investment vehicle to the detriment of all the other productive and pro-social uses for it.

Use it or lose it I say. You wanna invest? That's what the S&P and index funds are for. Go thusly