A database on Heroku and other similar offerings such as AppFog, AppHarbor, etc, are really expensive. I'd rather pay 15$/month starting out and install everything I need with simple commands - and I'm not even a Linux-guy! Imagine what a veteran Linux guy would do!
Don't you lose too much control? The last time I tried to deploy a Rails app on Heroku, I couldn't figure out where to save and well, "admin" user uploads on my website. (I was building a 9Gag clone to practice) and it turns out you cannot save data on a Heroku app. You need to pay for YET ANOTHER service such as Amazon content hosting.
So for a full stack you're looking at paying about 80$/month, that includes Heroku, Database, and content hosting. For a lean, mean startup I'd rather just pay 15$ for a VPS and handle things my way.
It's not like it's that hard. If and WHEN I need a service such as Heroku for load balancing, etc when my app reaches critical mass I'll be able to afford it. Not before.