Insanely misleading title. The author is basically showing off their A100 GPUs. The ChatGPT part is just clickbait, the only relevance ia that ChatGPT supposedly is also using A100s (definitely not only eight of them though).
back
1 comments
the real question is whether ChatGPT needs that kind of hardware for a single user. a client side version.
Yes it does.
When stable diffusion came out, my 8gb gpu could handle 1 image at 512x512. Now i can generate 1000x800 with batch size 4. I wonder if Chatgpt has the same unused optimisation potential
Stable Diffusion is a lot smaller than GPT3. It has a bit less than a billion parameters while GPT3 has 175 billion parameters.
I would bet that we can do better than GPT3 with less computing resources but that would need a different model architecture.
> I wonder if Chatgpt has the same unused optimisation potential
Doubtful. As an example, OpenAI released Triton, a programming language explicitly for optimizing models. For the scale they're at they'd be crazy not to throw an engineer or two at optimizing the models they're going to send to production.
Make a Google of "hardware overhang"
Our brains outperform GPT-3 with only 30w of power. The potential for software optimization may be many orders of magnitude.
Wonder where is that 30w comes from. I found some said chess player used 6000 calories per day and that does not seem to tie in the number.
The brain mostly uses the same amount of calories all the time. It's the large muscles that can influence energy use.
what packages are you using?
I think I have DiffusionBee installed but I haven't updated it for months