Martin Casado from a16z stated an opinion that 80% of US startups are likely using less expensive open models, usually from China. Chamath P. on the All In Podcast said that his company is using Chinese models, but hosted in the US and he cited his company as a huge inference user.
back
1 comments
Important to distinguish that they're likely using them for internal workflows (agents, etc) where the scope is well defined and they can tune their prompts and evals to accommodate a lower-performance model.
Nobody is advocating switching their coding agents to open source (yet), but that's not the bulk of the tokens in companies that have automated workflows integrated into their business.