The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning

The paper proposes three innovative techniquesModel Update Decomposition (MUD), Block-wise Kronecker Decomposition (BKD), and Aggregation-Aware Decomposition (AAD)to enhance low-rank decomposition methods in federated learning, addressing key issues of what to decompose, how to decompose, and how to aggregate. Experimental results demonstrate that these methods lead to faster convergence and higher accuracy, supported by a theoretical analysis confirming the convergence of MUD.