At work we use cfengine/puppet to maintain systems, but we've started experimenting with Ansible to perform simple setup of hosts.

Personally I use Slaughter, which is serverless and applies policies located on github. (Disclaimer I wrote it). I consider it a server-less cfengine-lite, but with the added freedom of allowing arbitrary perl

http://steve.org.uk/Software/slaughter/

When it comes to any automation you need to set your limits, and decide what you're doing:

* Just setting up an initial stack of services, etc. * Maintaining the system going forward. Reverting local changes, etc.