Completely agree. These foundational models are more akin to the invention of transistors or the internet than anything else. It's a little disheartening from a startup perspective to see already large "incumbents" in the form of Google, Open-AI but I think most of the value created, in the future, from generative AI, will not be captured by these players.
back
1 comments
The question is if the models can get significantly better or are they already near the peak of their capabilities.
Researchers are still picking low hanging fruit by the fortnight. We are a lo00000ng way from a place where we may even estimate the peak capability.
Look at what GPT-4 can achieve from only receiving text and a primitive form of image training.
But models like GPT-4 depend on good quality training data, and that resource is limited, means maybe we aren't so long away from peak at least for LLMs.
There is no unlimited growth, and after the low hanging fruits each step in progress will be exponentially harder.
While true, we’ve been successfully keeping up this illusion for the whole economy for many decades.
> But models like GPT-4 depend on good quality training data, and that resource is limited
Most data has not been incorporated into a model yet. Most data is not in text format. We don't know what is possible once that changes.