The problem is due to the quartz oscillators these devices use, which are the same ones we use in phones, computers etc., which have the same problem as a result.
You don't notice this because just about every network-connected device these days uses NTP or something similar to keep its clock constantly up to date, but the clock itself is still inherently inaccurate.
There are also other mechanisms to keep clocks in sync by the way. Some mains-connected devices keep time using the 50Hz/60Hz mains voltage. Various countries have radio broadcasts that devices can be used to keep time (DE, US and JP run them, I believe).
Why nanoseconds though? Usually a sub-microsecond sync is good enough. I can only think of some specialized phase measurements, really, like when you need to know from which direction a radio signal is coming based on timing of two antennas.
(Edit: Bad math left my underestimating by a few order of magnitude. Whoops.)