After working in GPGPU for a while, I have become really wary of vendor claims. Vendors do all sort of tricks to make their claims look good : comparing against single core non-optimized CPU code, comparing different algorithms on GPU and CPU, not reporting the time required to transfer data between CPU and GPU, comparing only 32-bit float performance when 64-bit floats should be used and so forth.
Of course, GPUs are certainly very good for some types of problems. But they are not magical solutions and are not suitable for most problems.
The basic difference is that the silicon used for hiding memory latency (cache) in a CPU is used for more raw computational power and memory bandwidth in a GPU. So if the data are structured so they fit into the data-parallel paradigm, the GPU will kill a normal CPU. Not because of some magic, but because they were designed to do that.
You wouldn't try to run an operating system on a GPU, or a web browser, or whatever. But to say that a GPU only can handle rasterization is to vastly understate the realm of applicability. There are plenty of fp-intensive, data-parallel tasks that fit very well into a GPU paradigm. A lot of scientific calculations, for example.
Reminds me of: http://www.dilbert.com/2009-03-02/