FedLDP： Efficient Federated Learning with Localized Differential Privacy

doi:10.3969/j.issn.1006-2475.2025.09.016

Abstract

Abstract:
Abstract： Federated learning， as a distributed machine learning framework， allows users to collaboratively train models by sharing model parameters without disclosing raw data. However， model parameters may still contain a substantial amount of sensitive information， and direct sharing poses considerable threats to individuals’ privacy. The state-of-the-art solution for this problem is local differential privacy， which can resist adversaries with arbitrary background knowledge and protect private information thoroughly. Due to the high dimensionality and multi-round characteristics of federated learning parameters， it is particularly challenging to apply local differential privacy into federated learning. In this paper， we propose FedLDP， an efficient algorithm to privately federate learning. To avoid the individuals’ privacy leakage， in this algorithm， an exponential mechanism-based dimension selection is used to select important parameter dimensions for global aggregation， and Laplace mechanism is utilized to perturb the selected parameter dimensions. In addition， to improve the learning efficiency and overall performance of the model， an incremental privacy budget allocation strategy is designed to adjust the privacy budget allocation during the iteration process， optimizing the model training process. We theoretically prove that FedLDP satisfies [ε]-LDP， and extensive experiments using MINIST and Fashion-MINIST datasets demonstrate that FedLDP improves the final model’s accuracy by 5.07 percentage points and 3.01 percentage points under the same level of privacy constraints compared with state-of-the-art schemes.

Key words: Key words： incremental privacy budget allocation, differential privacy, dimension selection, federated learning

CLC Number:

中图分类号：TP391

CHENG Mengyuan, LI Yanhui, LYU Tianci, ZHAO Yuxin, HUANG Chen. FedLDP： Efficient Federated Learning with Localized Differential Privacy[J]. Computer and Modernization, 2025, 0(09): 109-118.

References

［1］ WANG X， LI J， LI J， et al. Multilevel similarity model for high-resolution remote sensing image registration［J］. Information Sciences， 2019，505（C）：294-305.
［2］ CAI Y F， LUAN T Y， GAO H B， et al. YOLOv4-5D： An effective and efficient object detector for autonomous driving［J］. IEEE Transactions on Instrumentation and Measurement， 2021，70（C）：1-13.
［3］ MCMAHAN B， MOORE E， RAMAGE D， et al. Communication
-efficient learning of deep networks from decentralized data［C］// Artificial Intelligence and Statistics. PMLR， 2017：1273-1282.
［4］ MELIS L， SONG C， DE CRISTOFARO E， et al. Exploiting unintended feature leakage in collaborative learning［C］// 2019 IEEE Symposium on Security and Privacy （SP）. IEEE， 2019：691-706.
［5］ NASR M， SHOKRI R， HOUMANSADR A. Comprehensive privacy analysis of deep learning： Passive and active white-box inference attacks against centralized and federated learning［C］// 2019 IEEE Symposium on Security and Privacy （SP）. IEEE， 2019：739-753.
［6］ XU G， LI H W， ZHANG Y， et al. Privacy-preserving federated deep learning with irregular users ［J］. IEEE Transactison on Depen-dable and Secure Computing， 2020，19（2）：1364-1381.
［7］ TRUEX S， BARACALDO N， ANWAR A， et al. A hybrid approach to privacy preserving federated learning ［C］// Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security. ACM， 2019：1-11.
［8］ YIN L H， FENG J Y， XUN H， et al. A privacy-preserving federated learning for multiparty data sharing in social IoTs［J］. IEEE Transactions on Network Science and Engineering， 2021，8（3）：2706-2718.
［9］ DWORK C. Differential privacy［C］// Automata， Languages and Programming： 33rd International Colloquium， ICALP 2006. Springer， 2006：1-12.
［10］ YANG M M， GUO T L， ZHU T Q， et al. Local differential privacy and its applications： A comprehensive survey［J］. Computer Standards & Interfaces2024，89（C）：103827-103847.
［11］ ZHAO Y， ZHAO J， YANG M M， et al. Local differential privacy based federated learning for internet of things［J］. IEEE Internet of Things Journal， 2021，8（11）：8836-8853.
［12］ MAHAWAGA A P C， BERTOK P， KHALIL I， et al. Local differential privacy for deep learning［J］. IEEE Internet Things， 2020，7（7）：5827-5842.
［13］ JIA W， SUN M， LIAN J， et al. Feature dimensionality reduction： A review［J］. Complex & Intelligent Systems， 2022，8（3）：2663-2693.
［14］ WANG N， XIAO X K， YANG Y， et al. Collecting and analyzing multidimensional data with local differential privacy［C］// 2019 IEEE 35th International Conference on Data Engineering （ICDE）. IEEE， 2019：638-649.
［15］ WANG Y S， TONG Y X， SHI D Y. Federated latent dirichlet allocation： A local differential privacy based framework［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2020，34（4）：6283-6290.
［16］ SHIN H， KIM S， SHIN J， et al. Privacy enhanced matrix factorization for recommendation with local differential privacy［J］. IEEE Transactions on Knowledge and Data Engineering， 2018，30（9）：1770-1782.
［17］ BASSIL R， SMITH A， THAKURTA A. Private empirical risk minimization： Efficient algorithms and tight error bounds［C］// 2014 IEEE 55th Annual Symposium on Foundations of Computer Science. IEEE， 2014：464-473.
［18］ WANG J Z， KONG L W， HUANG Z， et al. Research review of federated learning algorithms［J］. Big Data Research， 2020，6（6）：64-82.
［19］ WANG B L， YAO Y S， SHAN S， et al. Neural cleanse： Identifying and mitigating backdoor attacks in neural networks［C］// 2019 IEEE Symposium on Security and Privacy （SP）. IEEE， 2019：707-723.
［20］ SHOKRI R， SHMATIKOV V. Privacy-preserving deep learning［C］// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM， 2015：1310-1321.
［21］ ABADI M， CHU A， GOODFELLOW I， et al. Deep learning with differential privacy［C］// Proceedings of the 2016 ACM SIG-SAC Conference on Computer and Communications Security. ACM， 2016：308-318.
［22］ GEYER R C， KLEIN T， NABI M. Differentially private federated learning： A client level perspective［J］. arXiv preprint arXiv：1712.07557， 2017.
［23］ MIAO Y B， XIE R P， LI X H， et al. Compressed federated learning based on adaptive local differential privacy［C］// Proceedings of the 38th Annual Computer Security Applications Conference. ACM， 2022：159-170.
［24］ CUI L Z， MA J， ZHOU Y P， et al. Boosting accuracy of differentially private federated learning in industrial IoT with sparse responses［J］. IEEE Transactions on Industrial Informatics， 2022，19（1）：910-920.
［25］ WANG B C， CHEN Y G， JIANG H， et al. PPeFL： Privacy-preserving edge federated learning with local differential privacy［J］. IEEE Internet of Things Journal， 2023，10（17）：15488-15500.
［26］ LIU R X， CAO Y， YOSHIKAWA M， et al. Fedsel： Federated sgd under local differential privacy with top-k dimension selection［C］// The 25th International Conference on Database Systems for Advanced Applications. Springer， 2020：485-501.
［27］ JIANG X， ZHOU X B， GROSSKLAGS J. SignDS-FL： Local differentially private federated learning with sign-based dimension selection［J］. ACM Transactions on Intelligent Systems and Technology （TIST）， 2022，13（5）：90-111.
［28］ LEE J， KIFER D. Concentrated differentially private gradient descent with adaptive per-iteration privacy budget［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM， 2018：1656-1665.
［29］ LIU C， TIAN Y L， TANG J C， et al. A novel local differential privacy federated learning under multi-privacy regimes［J］. Expert Systems with Applications， 2023，227（C）：120266-120281.
［30］ DONG F， GE X H， LI Q， et al. PADP-FedMeta： A personalized and adaptive differentially private federated meta learning mechanism for AIoT［J］. Journal of Systems Architecture， 2023，134（C）：102754-102769.
［31］ RASHID T. Make Your Own Neural Network［M］. CreateSpace Independent Publishing Platform， 2016.
［32］ LECUN Y， BOTTOU L， BENGIO Y， et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE， 1998，86（11）：2278-2324.
［33］ XIAO H， RASUL K， VOLLGRAF R. Fashion-MNIST： A novel image dataset for benchmarking machine learning algorithms［J］. arXiv preprint arXiv：1708.07747， 2017.

[1]	CHENG Yuwen, JING Yijun, SHI Zicheng, JING Changqiang, GUO Feng, WU Chuankun. A Data Privacy Protection Scheme Based on Federated Learning [J]. Computer and Modernization, 2025, 0(08): 63-69.
[2]	YUAN Hongwei1, CHANG Lijun1, HAO Jianhuan2, FAN Na2, WANG Chao2, LUO Chuang2, ZHANG Zehui2. Trajectory Interest Points Mining Based on Label Propagation and Privacy Protection [J]. Computer and Modernization, 2024, 0(05): 46-54.
[3]	YANG Ju, DENG Zhiliang, YANG Zhiqiang, WANG Yan, ZHAO Zhongyuan. Decentralized Federation Learning Based on Fed-DPDOBO [J]. Computer and Modernization, 2024, 0(04): 99-106.
[4]	FENG Li-gang, ZHU You-wen, . Utility-optimized Local Differential Privacy Mechanism for Protecting Location Privacy [J]. Computer and Modernization, 2022, 0(09): 99-105.
[5]	ZHAO Long, LONG Shi-gong, . Crowdsourcing Privacy Protection Method to Local Differential Privacy [J]. Computer and Modernization, 2021, 0(07): 115-119.
[6]	CHEN Yang， YU Shoujian. Research on Differential Privaty for Decision Tree Release Technology [J]. Computer and Modernization, 2017, 0(3): 59-.