MAP: Revisiting Weight Decomposition for Low-Rank Adaptation

The paper introduces MAP, a new framework for parameter-efficient fine-tuning of large language models that rigorously decomposes weight adaptation into direction and magnitude by representing weight matrices as high-dimensional vectors, enabling more interpretable and flexible updates. Extensive experiments demonstrate that MAP enhances existing PEFT methods like LoRA, offering a simple, universal approach that can serve as a default for future fine-tuning strategies.