一种自适应系统规模的联邦深度学习方法

A Federated Deep Learning Method for Adaptive System Scale

摘要: 在面向复杂任务协作的环境中，通信带宽和计算资源的限制以及对隐私保护的需求共同构成了该研究领域的主要挑战。为了解决这些问题，研究者提出了联邦学习(Federated Learning, FL)框架作为一种解决方案。FL允许多个设备在不直接交换原始数据的情况下进行协同模型训练，从而降低了通信需求并保护了数据隐私。然而，一些FL方法采用了全客户端参与的策略，即所有客户端在每一轮中更新其本地模型。这种方法不仅增加了通信次数，而且随着客户端规模的增大，也会导致系统性能下降、响应延迟等问题。因此，本文介绍了一种基于原始–对偶优化的新FL协议(Federated Deep Learning Alternating Direction Method of Multipliers, FDLADMM)。FDLADMM算法利用双变量来引导客户端进行本地训练，减少了设备间的通信次数，优化了模型训练速度，并且随着系统规模增大，无需进行超参数调整即可有效适应。通过实验，本文展示了所提出的方法在通信效率和训练速度方面的优势，并且当系统规模不断调整时，无需进行超参数调整即可有效适应。这一创新的方法为应对复杂任务协作中的挑战提供了一种可行且高效的解决方案，并有望在未来的研究和实践中得到广泛应用。

Abstract: In the context of complex task-oriented collaboration, the limitations of communication bandwidth and computing resources, as well as the need for privacy protection, together constitute major challenges in this research area. To solve these problems, researchers propose a framework of Federated Learning (FL) as a solution. FL allows multiple devices to perform collaborative model training without directly exchanging raw data, thereby reducing communication requirements and protecting data privacy. However, some FL methods adopt a full-client-participation strategy, where all clients update their local model in each round. This method not only increases the number of communication, but also leads to performance degradation and response delay with the increase of client size. Therefore, this paper introduces a new FL protocol (Federated Deep Learning Alternating Direction Method of Multipliers, FDLADMM) based on primitive-dual optimization. FDLADMM algorithm uses bivariate to guide the client to carry out local training, reduces the communication times between devices, optimizes the training speed of the model, and with the increase of the system size, it can adapt effectively without super parameter adjustment. Through experiments, the advantages of the proposed method in communication efficiency and training speed are demonstrated, and it can be adapted effectively without hyperparameter adjustment when the system size is adjusted continuously. This innovative approach provides a viable and efficient solution to the challenges of collaboration in complex missions and is expected to be widely applied in future research and practice.