If you’re concerned about tracking as a website operator you can simply load the fonts from your own server, most of them can be found on Github and have liberal licenses (I think all fonts on fonts.google.com)
It's been suggested that Browsers or Operating Systems choose a useful X value and install the top fonts everywhere like they used to install the classic "core web fonts".
https://git.synz.io/Synzvato/decentraleyes
Web developers who are concerned about privacy exposure from Google Fonts can use google-webfonts-helper to extract the font files and self-host them.
Note that some browsers are going in the opposite direction and actively hiding installed fonts from websites due to the potential for fingerprinting.
Different machines render fonts in a unique way. Consider different GPUs (hardware differences), different drivers (kernel/driver differences), browsers (user-land differences), and sizes (user settings) can all contribute to a font being rendered differently per-user and increasing entropy of a particular unique font id.