I spent 2 ours this morning playing with RDS - not really a waste of time since I deploy most of my customer's projects to EC2, etc. Sort-of kicking the tires.
Anyway, it is easy to use, and not having to deal yourself with failure recover, managing a EBS volume if you run MySQL yourself, etc. makes it look like a good service once it comes out of beta.
A few things: right now, you can't pay a reservation fee to get the really cheap EC2 rates. (Basically RDS is a managed-for-your EC2+EBS.) Also, it would be great if they also support PostgreSQL.