I worked on the LHC's CMS data team about 2-3 years ago. We had thousands of machines crunching data. Comparing the cost to Amazon, we laughed at the deal we were getting with disposable hardware (compared to EC2 pricing). Even buying in bulk from Amazon, the dedicated hardware was cheaper.

Amazon is a huge premium over physical hardware. You use them when a) you want to scale immediately, b) you have financial reasons for not buying your own gear for long-term use (1-3 years), and c) you don't mind paying the premium.