Thursday, March 20, 2025

LoRA


https://arxiv.org/abs/2106.09685

https://huggingface.co/blog/lora


W(d*d) = A(d*r) B(r*d)

r << d, ex: d=512, r=4 (ie. rank) 

運算大幅下降... (W不動, 僅訓練AB)





























No comments:

Related Posts Plugin for WordPress, Blogger...