| Title | Privacy-Preserving Policy Synthesis in Markov Decision Processes |
| Publication Type | Conference Paper |
| Year of Publication | 2020 |
| Authors | Gohari, Parham, Hale, Matthew, Topcu, Ufuk |
| Conference Name | 2020 59th IEEE Conference on Decision and Control (CDC) |
| Date Published | Dec. 2020 |
| Publisher | IEEE |
| ISBN Number | 978-1-7281-7447-1 |
| Keywords | Computing Theory, Computing Theory and Privacy, control theory, Differential privacy, dynamic programming, Heuristic algorithms, History, Human Behavior, Markov processes, privacy, Privacy Policies, Probabilistic logic, pubcrawl, resilience, Resiliency, Scalability |
| Abstract | In decision-making problems, the actions of an agent may reveal sensitive information that drives its decisions. For instance, a corporation's investment decisions may reveal its sensitive knowledge about market dynamics. To prevent this type of information leakage, we introduce a policy synthesis algorithm that protects the privacy of the transition probabilities in a Markov decision process. We use differential privacy as the mathematical definition of privacy. The algorithm first perturbs the transition probabilities using a mechanism that provides differential privacy. Then, based on the privatized transition probabilities, we synthesize a policy using dynamic programming. Our main contribution is to bound the "cost of privacy," i.e., the difference between the expected total rewards with privacy and the expected total rewards without privacy. We also show that computing the cost of privacy has time complexity that is polynomial in the parameters of the problem. Moreover, we establish that the cost of privacy increases with the strength of differential privacy protections, and we quantify this increase. Finally, numerical experiments on two example environments validate the established relationship between the cost of privacy and the strength of data privacy protections. |
| URL | https://ieeexplore.ieee.org/document/9304015 |
| DOI | 10.1109/CDC42340.2020.9304015 |
| Citation Key | gohari_privacy-preserving_2020 |
Privacy-Preserving Policy Synthesis in Markov Decision Processes