Decentralized Federation Learning Based on Fed-DPDOBO

doi:10.3969/j.issn.1006-2475.2024.04.017

Abstract

Abstract:
Abstract： The traditional client-server architecture federation learning is an effective means of solving the problem of data silos， where the central server is under enormous bandwidth pressure and the decentralized peer-to-peer architecture federation learning improves this situation to some extent. However， clients of federal learning also suffer from the risk of data privacy breaches and the gradient information of their cost function is difficult to obtain in some cases. To address these issues， this paper designs an Federated Differential Privacy Distributed One-point Bandit Online algorithm （Fed-DPDOBO） for peer-to-peer architecture federation learning under consistency constraints， which effectively addresses the problems of bandwidth limitation of the central node and unknown gradient information of the client. In addition， data privacy for each client is well protected due to the use of differential privacy technology. Finally， the effectiveness of this paper's algorithm is verified by conducting decentralized federation learning experiments with the MINST dataset.

Key words: Key words： data silos, federated learning, consistency constraints, peer-to-peer architecture, differential privacy, one-point Bandit

CLC Number:

TP309

YANG Ju, DENG Zhiliang, YANG Zhiqiang, WANG Yan, ZHAO Zhongyuan. Decentralized Federation Learning Based on Fed-DPDOBO[J]. Computer and Modernization, 2024, 0(04): 99-106.

References

［1］罗向龙，焦琴琴，牛力瑶，等. 基于深度学习的短时交通流预测［J］. 计算机应用研究， 2017，34（1）：91-93.
［2］ TIAN Z H， GAO X S， SU S， et al. Evaluating reputation management schemes of Internet of vehicles based on evolutionary game theory［J］. IEEE Transactions on Vehicular Technology， 2019，68（6）：5971-5980.
［3］周润佳. 基于深度学习的股票趋势预测算法［J］. 计算机与现代化， 2023（1）：69-73.
［4］ QI E N， DENG M. R&D investment enhance the financial performance of company driven by big data computing and analysis［J］. International Journal of Computer Systems Science and Engineering， 2019，34（4）：237-248.
［5］ ZHANG J H， YIN Z， CHEN P， et al. Emotion recognition using multi-modal data and machine learning techniques： A tutorial and review［J］. Information Fusion， 2020，59：103-126.
［6］ YANG Q， LIU Y， CHEN T J， et al. Federated machine learning： Concept and applications［J］. ACM Transactions on Intelligent Systems and Technology （TIST）， 2019，10（2）. DOI： 10.1145/3298981.
［7］杨强. AI与数据隐私保护：联邦学习的破解之道［J］. 信息安全研究， 2019，5（11）：961-965.
［8］王健宗，孔令炜，黄章成，等. 联邦学习算法综述［J］. 大数据， 2020，6（6）：64-82.
［9］ LIM H K， KIM J B， ULLAH I， et al. Federated reinforcement learning acceleration method for precise control of multiple devices［J］. IEEE Access， 2021，9：76296-76306.
［10］ WANG S Q， TUOR T， SALONIDIS T， et al. Adaptive federated learning in resource constrained edge computing systems［J］. IEEE Journal on Selected Areas in Communications， 2019，37（6）：1205-1221.
［11］张斯杰. 基于同态加密与RingAllreduce的去中心化联邦学习［J］. 电脑知识与技术， 2021，17（34）：25-27.
［12］ MA J， ZHANG Q C， LOU J， et al. Communication efficient federated generalized tensor factorization for collaborative health data analytics［C］// Proceedings of the Web Conference 2021. ACM， 2021：171-182.
［13］王晋东，张明清，韩继红. 信息系统安全技术策略研究［J］. 计算机应用研究， 2001，18（5）：61-63.
［14］ PHONG L T， AONO Y， HAYASHI T， et al. Privacy-preserving deep learning via additively homomorphic encryption［J］. IEEE Transactions on Information Forensics and Security， 2018，13（5）：1333-1345.
［15］ DWORK C. Differential privacy［C］// Proceedings of the 33rd International Colloquium on Automata， Languages and Programming （ICALP）. Springer， 2006：1-12.
［16］ WEI K， LI J， DING M， et al. Federated learning with differential privacy： Algorithms and performance analysis［J］. IEEE Transactions on Information Forensics and Security， 2020，15：3454-3469.
［17］ TRUEX S， LIU L， CHOW K H， et al. LDP-Fed： Federated learning with local differential privacy［C］// Proceedings of the 3rd ACM International Workshop on Edge Systems， Analytics and Networking. ACM， 2020：61-66.
［18］ WANG C， XU S Y， YUAN D M， et al. Push-sum distributed online optimization with bandit feedback［J］. IEEE Transactions on Cybernetics， 2022，52（4）：2263-2273.
［19］ PANG Y P， HU G Q. Randomized gradient-free distributed optimization methods for a multiagent system with unknown cost function［J］. IEEE Transactions on Automatic Control， 2020，65（1）：333-340.
［20］ YAN J E， WANG Y， LI W L. Behavior sequence mining model based on local differential privacy［J］. IEEE Access， 2020，8：196086-196093.
［21］ FLAXMAN A D， KALAI A T， MCMAHAN H B. Online convex optimization in the bandit setting： Gradient descent without a gradient［C］// Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete Algorithms. ACM， 2005：385-394.
［22］ XI C G， KHAN U A. Distributed subgradient projection algorithm over directed graphs［J］. IEEE Transactions on Automatic Control， 2017，62（8）：3986-3992.
［23］ NEDIC A， OZDAGLAR A， PARRILO P A. Constrained consensus and optimization in multi-agent networks［J］. IEEE Transactions on Automatic Control， 2010，55（4）：922-938.
［24］ ISLAM M J， AHMAD S， HAQUE F， et al. Application of min-max normalization on subject-invariant EMG pattern recognition［J］. IEEE Transactions on Instrumentation and Measurement， 2022，71. DOI： 10.1109/TIM.2022.3220286.
［25］ ZHANG Z， ZHANG Y， GUO D， et al. Communication-efficient federated continual learning for distributed learning system with non-IID data［J］. Science China Information Sciences， 2023，66. DOI： 10.1007/s11432-020-3419-4.
［26］ YOSHIDA N， NISHIO T， MORIKURA M， et al. Hybrid-FL for wireless networks： Cooperative learning mechanism using non-IID data［C］// Proceedings of the 2020 IEEE International Conference on Communications （ICC）. IEEE， 2020. DOI： 10.1109/ICC40277.2020.9149323.
［27］ LI X C， ZHAN D C. FedRS： Federated learning with restricted softmax for label distribution non-IID data［C］// Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM， 2021：995-1005.

[1]	YUAN Hongwei1, CHANG Lijun1, HAO Jianhuan2, FAN Na2, WANG Chao2, LUO Chuang2, ZHANG Zehui2. Trajectory Interest Points Mining Based on Label Propagation and Privacy Protection [J]. Computer and Modernization, 2024, 0(05): 46-54.
[2]	LIANG Tian-kai, HUANG Kang-hua, LIU Kai-hang, LAN Lan, ZENG Bi. Deep Federated Image Classification Method Based on Bilateral Homomorphic Encryption [J]. Computer and Modernization, 2023, 0(12): 36-40.
[3]	FENG Li-gang, ZHU You-wen, . Utility-optimized Local Differential Privacy Mechanism for Protecting Location Privacy [J]. Computer and Modernization, 2022, 0(09): 99-105.
[4]	ZHAO Long, LONG Shi-gong, . Crowdsourcing Privacy Protection Method to Local Differential Privacy [J]. Computer and Modernization, 2021, 0(07): 115-119.
[5]	CHEN Yang， YU Shoujian. Research on Differential Privaty for Decision Tree Release Technology [J]. Computer and Modernization, 2017, 0(3): 59-.