Using an alternative connection to activate the protocol is probably the only reasonable defense:
* An https service that can enable a port for connections from an IP for a limited time (which has some security implications too since it would need root access, or to trigger something with root access)
* Something built into KLO hardware, or software provided by your provider (including something simple like turn on the firewall a few minutes after boot, and using the reboot trigger and connecting in that brief window.. as long as you don't mind your server down - probably ok as an emergency recovery strategy)
* Some form of port knocking