[I work for a hosting company.]

Magento is very very very heavyweight. We've had the best success with running it behind varnish, via the varnish plugin.

Other minor wins involve using memcached for caching, and applying the tweaks you mention.