I think we'll get there but it will take years to arrive at a robust solution, because of the complexity of the ecosystem. The cooperative matrix extension, the portable and standard way to do what is usually marketed as "tensor cores," just landed in the spec and drivers a couple months ago. It will take people time to figure out how to use them effectively, partly because you have to query which options are supported at runtime.

There's still a driver and you're still at mercy of it (bugs and all), it's just much more likely to be installed on your Windows or Linux machine than a ROCm stack.