OK, apparently this is not such a problem; from the book: "At the time of writing, CouchDB doesn’t remember continuous replications over a server restart. For the time being, you are required to trigger them again, when you restart CouchDB."

So, put something in /etc/rc.local for each server to start the replication again.