Do you use VPN? This is a common misconfiguration of a server-side NAT related to too long or too short NAT timeouts combined with "act like a blackhole if we don't know anything about this connection".