异构网络下一种带松弛步的联邦学习算法

A Federated Learning Algorithm with a Relaxation Step for Heterogeneous Networks

摘要: 联邦学习是一个在保护用户隐私、数据安全方面具有显著优势的机器学习框架。针对联邦学习中各个设备间存在的数据异质性与系统异质性，许多学者提出了相应的解决方案，如FedProx算法等。考虑到松弛步可以起到提高收敛速度的作用，本文我们在FedProx的基础上引入松弛步，提出一种带松弛步的联邦学习算法FedProx + Relaxation。本文对FedProx + Relaxation的收敛性进行了理论分析，并通过数值实验展示了该算法的有效性与稳健性。通过数值实验，本文说明了FedProx + Relaxation相比于FedProx具有更加稳健的收敛效果。

Abstract: Federated learning is a machine learning framework that has significant advantages in protecting user privacy and data security. Considering the data heterogeneity and system heterogeneity between various devices in federated learning, many scholars have proposed corresponding solutions, such as FedProx algorithm, etc. Considering that the relaxation step can play a role in improving the speed of convergence, a relaxation step on the basis of the FedProx algorithm was introduced. And then a federated learning algorithm with a relaxation step, i.e., FedProx + Relaxation is proposed here. The convergence of the FedProx + Relaxation is analyzed theoretically, the efficiency and robustness of the algorithm are demonstrated by numerical experiments. Through the numerical experiment, it shows that the FedProx + Relaxation has more robust convergence than FedProx.