Security by Obscurity is great. This is a takeaway from the port knocking conversation here last week. If there's a zero day exploit in sshd, I'd rather it be behind some layer you would have to "know" to get in, rather than sitting open to the world. Why make your target bigger than it needs to be?