Last week I finally bit the bullet and bought a VPS from DigitalOcean and installed the bits and kibbles needed for Rails apps. Specifically Nginx and Passenger. Was it easy? No - but now I have total control over my machine, and my app (which let's face it, will probably never need to scale for millions of users at a time) is in my complete control.
Heroku doesn't even have a local filesystem, what's up with that?