FedLDP：本地化差分隐私下一种高效联邦学习方法

doi:10.3969/j.issn.1006-2475.2025.09.016

摘要/Abstract

摘要：
摘要：联邦学习作为一种分布式机器学习框架，允许用户在不泄露原始数据的情况下通过共享模型参数来协作训练模型。然而，模型参数仍然可能包含大量隐私敏感信息，直接对其共享存在泄露用户隐私信息的风险。本地化差分隐私能够抵御具有任意背景知识的攻击者，为隐私信息提供更全面的保护。由于联邦学习的参数数据高维度和多轮次的特点，给本地化差分隐私应用于联邦学习带来了挑战。为此，本文提出一种满足本地化差分隐私的联邦学习算法FedLDP。该算法利用维度选择策略（EMDS）挑选出用于全局聚合的重要参数维度；采用拉普拉斯机制扰动所选的参数维度；为了提高模型的学习效率和整体性能，设计增量隐私预算分配策略调整迭代过程中的隐私预算分配方式，优化模型训练过程。理论分析证明FedLDP算法满足[ε]-本地化差分隐私。实验结果表明，在MNIST和Fashion-MNIST数据集上，FedLDP算法能够在相同级别的隐私约束下使模型准确率分别提升5.07百分点和3.01百分点，优于现有方法。

关键词: 关键词：增量隐私预算分配, 差分隐私, 维度选择, 联邦学习

Abstract:
Abstract： Federated learning， as a distributed machine learning framework， allows users to collaboratively train models by sharing model parameters without disclosing raw data. However， model parameters may still contain a substantial amount of sensitive information， and direct sharing poses considerable threats to individuals’ privacy. The state-of-the-art solution for this problem is local differential privacy， which can resist adversaries with arbitrary background knowledge and protect private information thoroughly. Due to the high dimensionality and multi-round characteristics of federated learning parameters， it is particularly challenging to apply local differential privacy into federated learning. In this paper， we propose FedLDP， an efficient algorithm to privately federate learning. To avoid the individuals’ privacy leakage， in this algorithm， an exponential mechanism-based dimension selection is used to select important parameter dimensions for global aggregation， and Laplace mechanism is utilized to perturb the selected parameter dimensions. In addition， to improve the learning efficiency and overall performance of the model， an incremental privacy budget allocation strategy is designed to adjust the privacy budget allocation during the iteration process， optimizing the model training process. We theoretically prove that FedLDP satisfies [ε]-LDP， and extensive experiments using MINIST and Fashion-MINIST datasets demonstrate that FedLDP improves the final model’s accuracy by 5.07 percentage points and 3.01 percentage points under the same level of privacy constraints compared with state-of-the-art schemes.

Key words: Key words： incremental privacy budget allocation, differential privacy, dimension selection, federated learning

中图分类号:

中图分类号：TP391

成梦圆, 李艳辉, 吕天赐, 赵玉鑫, 黄臣. FedLDP：本地化差分隐私下一种高效联邦学习方法[J]. 计算机与现代化, 2025, 0(09): 109-118.

CHENG Mengyuan, LI Yanhui, LYU Tianci, ZHAO Yuxin, HUANG Chen. FedLDP： Efficient Federated Learning with Localized Differential Privacy[J]. Computer and Modernization, 2025, 0(09): 109-118.

参考文献

［1］ WANG X， LI J， LI J， et al. Multilevel similarity model for high-resolution remote sensing image registration［J］. Information Sciences， 2019，505（C）：294-305.
［2］ CAI Y F， LUAN T Y， GAO H B， et al. YOLOv4-5D： An effective and efficient object detector for autonomous driving［J］. IEEE Transactions on Instrumentation and Measurement， 2021，70（C）：1-13.
［3］ MCMAHAN B， MOORE E， RAMAGE D， et al. Communication
-efficient learning of deep networks from decentralized data［C］// Artificial Intelligence and Statistics. PMLR， 2017：1273-1282.
［4］ MELIS L， SONG C， DE CRISTOFARO E， et al. Exploiting unintended feature leakage in collaborative learning［C］// 2019 IEEE Symposium on Security and Privacy （SP）. IEEE， 2019：691-706.
［5］ NASR M， SHOKRI R， HOUMANSADR A. Comprehensive privacy analysis of deep learning： Passive and active white-box inference attacks against centralized and federated learning［C］// 2019 IEEE Symposium on Security and Privacy （SP）. IEEE， 2019：739-753.
［6］ XU G， LI H W， ZHANG Y， et al. Privacy-preserving federated deep learning with irregular users ［J］. IEEE Transactison on Depen-dable and Secure Computing， 2020，19（2）：1364-1381.
［7］ TRUEX S， BARACALDO N， ANWAR A， et al. A hybrid approach to privacy preserving federated learning ［C］// Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security. ACM， 2019：1-11.
［8］ YIN L H， FENG J Y， XUN H， et al. A privacy-preserving federated learning for multiparty data sharing in social IoTs［J］. IEEE Transactions on Network Science and Engineering， 2021，8（3）：2706-2718.
［9］ DWORK C. Differential privacy［C］// Automata， Languages and Programming： 33rd International Colloquium， ICALP 2006. Springer， 2006：1-12.
［10］ YANG M M， GUO T L， ZHU T Q， et al. Local differential privacy and its applications： A comprehensive survey［J］. Computer Standards & Interfaces2024，89（C）：103827-103847.
［11］ ZHAO Y， ZHAO J， YANG M M， et al. Local differential privacy based federated learning for internet of things［J］. IEEE Internet of Things Journal， 2021，8（11）：8836-8853.
［12］ MAHAWAGA A P C， BERTOK P， KHALIL I， et al. Local differential privacy for deep learning［J］. IEEE Internet Things， 2020，7（7）：5827-5842.
［13］ JIA W， SUN M， LIAN J， et al. Feature dimensionality reduction： A review［J］. Complex & Intelligent Systems， 2022，8（3）：2663-2693.
［14］ WANG N， XIAO X K， YANG Y， et al. Collecting and analyzing multidimensional data with local differential privacy［C］// 2019 IEEE 35th International Conference on Data Engineering （ICDE）. IEEE， 2019：638-649.
［15］ WANG Y S， TONG Y X， SHI D Y. Federated latent dirichlet allocation： A local differential privacy based framework［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2020，34（4）：6283-6290.
［16］ SHIN H， KIM S， SHIN J， et al. Privacy enhanced matrix factorization for recommendation with local differential privacy［J］. IEEE Transactions on Knowledge and Data Engineering， 2018，30（9）：1770-1782.
［17］ BASSIL R， SMITH A， THAKURTA A. Private empirical risk minimization： Efficient algorithms and tight error bounds［C］// 2014 IEEE 55th Annual Symposium on Foundations of Computer Science. IEEE， 2014：464-473.
［18］ WANG J Z， KONG L W， HUANG Z， et al. Research review of federated learning algorithms［J］. Big Data Research， 2020，6（6）：64-82.
［19］ WANG B L， YAO Y S， SHAN S， et al. Neural cleanse： Identifying and mitigating backdoor attacks in neural networks［C］// 2019 IEEE Symposium on Security and Privacy （SP）. IEEE， 2019：707-723.
［20］ SHOKRI R， SHMATIKOV V. Privacy-preserving deep learning［C］// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM， 2015：1310-1321.
［21］ ABADI M， CHU A， GOODFELLOW I， et al. Deep learning with differential privacy［C］// Proceedings of the 2016 ACM SIG-SAC Conference on Computer and Communications Security. ACM， 2016：308-318.
［22］ GEYER R C， KLEIN T， NABI M. Differentially private federated learning： A client level perspective［J］. arXiv preprint arXiv：1712.07557， 2017.
［23］ MIAO Y B， XIE R P， LI X H， et al. Compressed federated learning based on adaptive local differential privacy［C］// Proceedings of the 38th Annual Computer Security Applications Conference. ACM， 2022：159-170.
［24］ CUI L Z， MA J， ZHOU Y P， et al. Boosting accuracy of differentially private federated learning in industrial IoT with sparse responses［J］. IEEE Transactions on Industrial Informatics， 2022，19（1）：910-920.
［25］ WANG B C， CHEN Y G， JIANG H， et al. PPeFL： Privacy-preserving edge federated learning with local differential privacy［J］. IEEE Internet of Things Journal， 2023，10（17）：15488-15500.
［26］ LIU R X， CAO Y， YOSHIKAWA M， et al. Fedsel： Federated sgd under local differential privacy with top-k dimension selection［C］// The 25th International Conference on Database Systems for Advanced Applications. Springer， 2020：485-501.
［27］ JIANG X， ZHOU X B， GROSSKLAGS J. SignDS-FL： Local differentially private federated learning with sign-based dimension selection［J］. ACM Transactions on Intelligent Systems and Technology （TIST）， 2022，13（5）：90-111.
［28］ LEE J， KIFER D. Concentrated differentially private gradient descent with adaptive per-iteration privacy budget［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM， 2018：1656-1665.
［29］ LIU C， TIAN Y L， TANG J C， et al. A novel local differential privacy federated learning under multi-privacy regimes［J］. Expert Systems with Applications， 2023，227（C）：120266-120281.
［30］ DONG F， GE X H， LI Q， et al. PADP-FedMeta： A personalized and adaptive differentially private federated meta learning mechanism for AIoT［J］. Journal of Systems Architecture， 2023，134（C）：102754-102769.
［31］ RASHID T. Make Your Own Neural Network［M］. CreateSpace Independent Publishing Platform， 2016.
［32］ LECUN Y， BOTTOU L， BENGIO Y， et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE， 1998，86（11）：2278-2324.
［33］ XIAO H， RASUL K， VOLLGRAF R. Fashion-MNIST： A novel image dataset for benchmarking machine learning algorithms［J］. arXiv preprint arXiv：1708.07747， 2017.

[1]	程钰雯, 景义君, 时自成, 荆长强, 郭锋, 武传坤. 基于联邦学习的数据隐私保护方案[J]. 计算机与现代化, 2025, 0(08): 63-69.
[2]	袁红伟1, 常利军1, 郝家欢2, 樊娜2, 王超2, 罗闯2, 张泽辉2. 基于标签传播的轨迹兴趣点挖掘及隐私保护[J]. 计算机与现代化, 2024, 0(05): 46-54.
[3]	杨巨, 邓志良, 杨志强, 王燕, 赵中原. 基于Fed-DPDOBO的分散式联邦学习[J]. 计算机与现代化, 2024, 0(04): 99-106.
[4]	冯立刚, 朱友文, . 保护位置隐私的效用优化本地差分隐私机制[J]. 计算机与现代化, 2022, 0(09): 99-105.
[5]	赵龙, 龙士工, . 基于本地差分隐私的众包隐私保护方法[J]. 计算机与现代化, 2021, 0(07): 115-119.
[6]	陈杨，于守健. 基于差分隐私的决策树发布技术研究[J]. 计算机与现代化, 2017, 0(3): 59-.
[7]	夏小玲，刘慧艺. 面向数据流的差分隐私直方图发布[J]. 计算机与现代化, 2016, 0(2): 52-57.