back

by kristjansson·4y ago·view on hn ↗
The causal relation to WFH is ... tenuous ... but now can see how losing all internal comms to an incident could make managers more nervous about WFH.

Of course announcing a policy change now is closing barn doors behind horses.

1 comments
When I was at Amazon in 2001 outages were done via everyone sitting at their desks on a POTS conference call. The engineers working the issue would be spread out around building at least, and the people who were actually on-call would generally be at home. Remote hands were often in a datacenter that was on the other side of continent.

The whole idea of piling everyone into a "war room" doesn't really scale anyway.

The bigger problem here was just no out of band comms.

Yeah, knocking out their backup IRC (or maybe it was the authentication to the backup IRC?) along with everything else was the key.
Word-of-mouth I've heard is that the backup IRC server was off-site working fine. The problem was that, because it was only used in emergencies and wasn't hooked into FB's central authentication, a lot of employees simply didn't remember how to log in to it.