In practice there are not so many. And e.g. on Windows I use an old VS compiler version and can then also use it with later VS versions (but not vice versa). Linux is also no issue because Qt is usually part of the distribution, i.e. there is no need to compile it as long as you depend on the standard version (not a custom one).
Personally I don't work with package managers, but prefer so setup and control the environment myself. It's not that difficult, and there is also Docker.