Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-Box Inference Attacks against Centralized and Federated Learning
Title | Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-Box Inference Attacks against Centralized and Federated Learning |
Publication Type | Conference Paper |
Year of Publication | 2019 |
Authors | Nasr, Milad, Shokri, Reza, Houmansadr, Amir |
Conference Name | 2019 IEEE Symposium on Security and Privacy (SP) |
Date Published | May 2019 |
Publisher | IEEE |
ISBN Number | 978-1-5386-6660-9 |
Keywords | active membership inference attacks, active white-box inference attacks, black-box attacks, centralized learning, CIFAR dataset, composability, comprehensive privacy analysis, Computational modeling, Data models, data privacy, Deep Learning, deep learning models, deep neural networks, Deep-learning, federated learning setting, Federated-Learning, gradient methods, Inference algorithms, inference mechanisms, Inference-Attacks, learning (artificial intelligence), Membership-Inference, Metrics, neural nets, privacy, privacy leakage, pubcrawl, resilience, Resiliency, stochastic gradient descent algorithm, Stochastic processes, Stochastic-Gradient-Descent, Training, Training data, white box cryptography, White Box Security, white-box membership inference attacks |
Abstract | Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the parameter updates of models during training. We design inference algorithms for both centralized and federated learning, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference attacks against deep learning algorithms to trace their training data records. We show that a straightforward extension of the known black-box attacks to the white-box setting (through analyzing the outputs of activation functions) is ineffective. We therefore design new algorithms tailored to the white-box setting by exploiting the privacy vulnerabilities of the stochastic gradient descent algorithm, which is the algorithm used to train deep neural networks. We investigate the reasons why deep learning models may leak information about their training data. We then show that even well-generalized models are significantly susceptible to white-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset. We also show how adversarial participants, in the federated learning setting, can successfully run active membership inference attacks against other participants, even when the global model achieves high prediction accuracies. |
URL | https://ieeexplore.ieee.org/document/8835245/ |
DOI | 10.1109/SP.2019.00065 |
Citation Key | nasr_comprehensive_2019 |
- Resiliency
- Inference-Attacks
- learning (artificial intelligence)
- Membership-Inference
- Metrics
- neural nets
- privacy
- privacy leakage
- pubcrawl
- resilience
- inference mechanisms
- stochastic gradient descent algorithm
- Stochastic processes
- Stochastic-Gradient-Descent
- Training
- Training data
- white box cryptography
- White Box Security
- white-box membership inference attacks
- data privacy
- active white-box inference attacks
- black-box attacks
- centralized learning
- CIFAR dataset
- composability
- comprehensive privacy analysis
- Computational modeling
- Data models
- active membership inference attacks
- deep learning
- deep learning models
- deep neural networks
- Deep-learning
- federated learning setting
- Federated-Learning
- gradient methods
- Inference algorithms