LISA: Layerwise Importance Sampling for Memory-Efficient LLM Fine-Tuning

An author claims better performance than LoRA in 50% of the time.

https://twitter.com/Rui45898440/status/1772996453557997924