https://arxiv.org/abs/2106.09685
https://huggingface.co/blog/lora
W(d*d) = A(d*r) B(r*d)
r << d, ex: d=512, r=4 (ie. rank)
運算大幅下降... (W不動, 僅訓練AB)
https://arxiv.org/abs/2106.09685
https://huggingface.co/blog/lora
W(d*d) = A(d*r) B(r*d)
r << d, ex: d=512, r=4 (ie. rank)
運算大幅下降... (W不動, 僅訓練AB)
No comments:
Post a Comment