One of the interesting variables in calculating ML training costs is developer time. The cost of a Data Scientist (or similar role) on an hourly basis will far outweigh the most expensive compute resource by several orders of magnitude. When you factor in time, the GPU immediately becomes more attractive. Other industries with heavy/time consuming computational workloads like CGI rendering have understood this for decades. It's difficult to attach a dollar sign to the value of speeding something up because it's not only about simply saving time itself but also about the way we work: Waiting around for results limits our ability to work iteratively, scheduling jobs becomes a project of its own, the process becomes less predictable etc.

Disclaimer: Paperspace team.