> Let's imagine that there is a huge amount of time dilation (we live on the surface of a neuron star or something).

That idea is the premise of https://en.wikipedia.org/wiki/Incandescence_(novel)