So, as I understand it, this is for finetuning a preexisting llm? So not actually training one from scratch. I guess that would be too much to ask for. Nonetheless, cheers to Jeremy and the gang for the work.
back
3 comments
For now, it's for finetuning.
The issue of to what degree it might be possible to train a model from scratch using QLoRA is still an open question. The relora paper showed that it can work in some situations, but attempts to scale it up were unsuccessful. The recent DoRA paper perhaps might allow a "re-DoRA" approach to work. If so, that could be combined with quantization to do "re-QDoRA"!
The headline and introduction on the linked page say "You can now train a 70b language model at home. We’re releasing an open source system, based on FSDP and QLoRA, that can train a 70b model on two 24GB GPUs."
How does "fine tuning" differ from "training?" Reading the linked article I had assumed I could create my own trained LLM at home with two 24GB GPUs.
The article actually sneaks in a footnote that answers this (https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html#fn1): "Throughout this article “training” can refer to either pre-training, or fine-tuning".
(Generally, we've told students at fast.ai since 2017 that they should almost never be starting from random weights -- most of the time it's best to start with a pretrained model and fine-tune that, even if it's from a somewhat different domain to the problem you're working on.)
Have you changed your mind on „The End of Finetuning“ (https://www.latent.space/p/fastai ) or did I simply misunderstand that?
Oh, and thanks for quirky stuff like your APL video!
The title of that podcast isn't something I actually said (IIRC). I commented in that interview that I feel we should not consider pre-training and fine-tuning to be as separate as we do now.
So you‘re generally in favor of mixing training data without separating them in phases, but when I use pretrained weights (as you recommend instead of random weights) I generally do not have access to whatever the neural net was pretrained with by someone else, so I have to make do with my finetuning data, yes?
Thank you!
Yes.
"The right way to fine-tune language models... is to actually throw away the idea of fine-tuning. There's no such thing. There's only continued pre-training."
:) i hope i didnt pervert your intent too too much for clickbait or something, i thought it was the spirit of what you said
You most definitely can, the main difference is that only partial ~2% of the parameters get updated during training. Say you start from a model like llama-70B which already knows english and has some world knowledge based on its pretraining dataset. It might not be ideal for drastic domain shifts, such as adapting a model to learn new languages (which might require a new tokenizer and model embeddings) but still might be possible to some extent.
Thank you for clarifying. I have been wanting to dip my toes into LLMs at home but obviously I have a steep learning curve ahead of me, and would need considerably beefier hardware!
It’s steep but manageable, absolutely go for it. The more people who understand the tech the better.
You can take an existing 70B model and train it to do a more specific task. You're teaching it the task but you're relying on a foundation model for the base understanding of the world/words/etc.
OK, that makes sense. Thank you!
Digging into the low rank structure of the gradients, instead of the weights seems like a promising direction for training from scratch with less memory requirements: https://twitter.com/AnimaAnandkumar/status/17656138151468933...
Simo linked some older papers with this same idea: https://twitter.com/cloneofsimo/status/1765796493955674286
Lit-GPT is what I have been using to pretrain models at home: https://github.com/Lightning-AI/litgpt
Using the openwebtext example, I can train a 700M param model to 2.6 loss in a few days on dual 4090s. Pretty awesome!
Training a 70b model from scratch uses 80,000 GPU hours (4,6 years if you have two of those GPUs).
The electricity would cost more than 10,000€ in Germany, just for the GPUs.