That's all too true. However it's important to remember that when H.264 started to get public traction (this is back in 2005, when Apple started pushing H.264 movie trailers), realtime CPU encoding was deemed impossible at that point in time. It just wasn't realistic back in 2005 to produce H.264 on a CPU, instead hardware encoders were used to accomplish the task.

Fast forward 10 years and basically every single laptop has hardware accelerated H.264 encoding through Intel's QuickSync FFHW and desktops have it through Nvidia. On top of that the software encoder (x264) is so fast that it's possible to do it in realtime on a CPU.

There is a fair chance that VP9 will see the exact same pattern here. Today CPU encoding is unfeasible, as you've noticed. However Intel is releasing VP9 encoding on their QuickSync encoder with newer CPU's and Nvidia will from my guess have it either 1 or 2 GPU generations from now. Same goes for AV1 here.

It's also interesting too think that the age of these aggressive improvements in CPU speed we've had over the past 14 years (from 2005) might come to an end, where efficiency instead comes from specialised hardware, such as fixed function hardware doing encoding on the chips, as NVENC/QuickSync does today. I don't know if realtime AV1 encoding will be feasible on a consumer grade CPU within the next 10 years, but i sure know it's hardware counterpart will be.