| 引用本文： |
-
罗玉玲,赵乾明,欧阳雪,付强,秦圣,刘俊秀,徐彦彦.基于Renyi差分隐私的神经网络自适应分层加噪框架[J].信息安全学报,已采用 [点击复制]
- Luo Yuling,Zhao Qianming,Ouyang Xue,Fu Qiang,Qin Sheng,Liu Junxiu,Xu Yanyan.Renyi Differential Privacy in Deep Neural Networks with Adaptive Hierarchical Noise Addition[J].Journal of Cyber Security,Accept [点击复制]
|
|
| 摘要: |
| 神经网络模型的训练过程中可能存在训练数据集中海量个人隐私信息的泄露的问题。差分隐私通过向模型训练过程引入随机扰动，为神经网络模型提供了具有严格数学保证的隐私保护机制。因此，采用差分隐私来保护神经网络中存在的用户个人隐私信息，以实现神经网络模型的隐私保障是有必要的。然而，现有差分隐私神经网络方法通常采用统一的噪声注入策略，难以根据不同网络层对模型输出的实际影响程度合理分配隐私预算；同时，传统隐私损失计算和组合方式在多轮迭代训练过程中存在计算复杂度较高、隐私损失计算不够精确的问题，并且模型隐私保护强度与模型可用性之间难以实现有效平衡。针对上述问题，本文提出了一种基于Rényi差分隐私的神经网络自适应分层加噪框架，旨在实现更精确的隐私预算分配和组合来减少噪声对模型性能的影响，达到模型可用性和隐私保障之间的相互平衡的目的。所提出的框架利用Wasserstein距离计算原始模型输出分布与对不同隐藏层进行扰动后的输出分布差异，以此衡量神经网络模型不同隐藏层对模型输出结果的贡献度。然后，根据隐藏层的贡献度大小分配隐私预算，向贡献度小的隐藏层权重梯度添加更多噪声，向贡献度大的隐藏层权重梯度添加较少噪声。最后，利用Rényi差分隐私对模型不同隐藏层的隐私预算进行，实现对模型总的隐私预算消耗进行精确计算。该框架在四个数据集和三种梯度更新算法上进行了实验，结果表明该框架在满足隐私保障的前提下，降低了噪声对模型性能的影响，有效提高了模型的准确率和精确率。并且，该框架能够在满足Rényi差分隐私约束的前提下，更高效地利用隐私预算，从而在模型性能与隐私保护强度之间实现了更优的权衡。 |
| 关键词: 神经网络模型 隐私保护 Renyi差分隐私 |
| DOI： |
| 投稿时间：2026-06-10修订日期：2026-09-23 |
| 基金项目: |
|
| Renyi Differential Privacy in Deep Neural Networks with Adaptive Hierarchical Noise Addition |
|
Luo Yuling, Zhao Qianming, Ouyang Xue, Fu Qiang, Qin Sheng, Liu Junxiu, Xu Yanyan
|
| (Guangxi Normal University) |
| Abstract: |
| During the training of neural network models, massive amounts of personal privacy information contained in training datasets may be exposed, resulting in potential privacy leakage. Differential privacy provides a privacy-preserving mechanism with rigorous mathematical guarantees by introducing random perturbations into the model training process. Therefore, incorporating differential privacy into neural network training is necessary to protect users'' personal privacy information and ensure the privacy of neural network models. However, existing differentially private neural network methods generally adopt uniform noise injection strategies, making it difficult to reasonably allocate privacy budgets according to the actual influence of different network layers on model outputs. Meanwhile, conventional privacy-loss accounting and composition methods may involve high computational complexity and insufficiently accurate priva-cy-loss estimation during multi-round iterative training. Consequently, it remains challenging to achieve an effective balance between the strength of model privacy protection and model utility. To address these issues, this paper proposes an Adaptive Hierarchical Rényi Differential Privacy (AHRDP) framework for neural networks, which aims to achieve more precise privacy-budget allocation and composition, thereby reducing the impact of noise on model performance and balancing model utility and privacy protection. The proposed framework employs the Wasserstein distance to meas-ure the discrepancy between the output distribution of the original model and those of the model after perturbing differ-ent hidden layers, thereby quantifying the contribution of different hidden layers to the model output. Based on the con-tribution of each hidden layer, the privacy budget is adaptively allocated such that more noise is injected into the weight gradients of hidden layers with lower contributions, while less noise is added to those of hidden layers with higher con-tributions. Finally, Rényi Differential Privacy (RDP) is employed to account for the privacy budgets assigned to different hidden layers, enabling accurate accounting of the total privacy-budget consumption of the model. Extensive experi-ments are conducted on four datasets using three gradient optimization algorithms. The results demonstrate that, while satisfying the required privacy protection, the proposed framework reduces the impact of noise on model performance and effectively improves classification accuracy and precision. Furthermore, under the constraints of Rényi Differential Privacy, the proposed framework utilizes the privacy budget more efficiently, thereby achieving a more effective trade-off between model performance and privacy protection strength. |
| Key words: neural network modeling,privacy preservation,Renyi difference privacy |
