Menu

Blog

Mar 9, 2024

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Posted by in category: computing

GaLore.

Memory-efficient LLM training by gradient low-rank projection.

V/@animaanandkumar.

For the first time, we show that the Llama 7B LLM can be trained on a single consumer-grade GPU (RTX 4090) with only 24GB memory.


Join the discussion on this paper page.

Comments are closed.