Securing Safety-Critical Machine Learning Algorithms - October 2018
PI(s), Co-PI(s), Researchers: Lujo Bauer, Matt Fredrikson (CMU), Mike Reiter (UNC)
HARD PROBLEM(S) ADDRESSED
This project addresses the following hard problems: developing security metrics and developing resilient architectures. Both problems are tackled in the context of deep neural networks, which are a particularly popular and performant type of machine learning algorithm. This project develops metrics that characterize the degree to which a neural-network-based classifier can be evaded through practically realizable, inconspicuous attacks. The project also develops architectures for neural networks that would make them robust to adversarial examples.
PUBLICATIONS
N/A this quarter
PUBLIC ACCOMPLISHMENT HIGHLIGHTS
We have made significant progress implementing a framework for explaining the predictions made by deep neural networks, and incorporating it into a graphical tool for use by researchers and practitioners. Explanations may identify the network-internal factors that cause misclassifications, and we leverage this capability to make progress on the hard problems above. We also believe that certain types of explanations can comprise a runtime defense, with a human in the loop, by exposing cases where predictions appear to be "made for the wrong reasons." Our approach to explanations allows analysts to parameterize queries of network behavior on the aspect being explained, the set of samples in question, and the portion of the network under study, and our tool gains flexibility by exposing these as options. While this tool is useful for our activities on the project, we plan to release it as an open-source project, as well as a more limited interactive web application, for other researchers as well.
COMMUNITY ENGAGEMENTS (If applicable)
Lujo Bauer presented work on adversarial machine learning at the C3E workshop in Atlanta in September. Matt Fredrikson began co-organizing a workshop on Security in Machine Learning, to be held in conjunction with the 2018 Neural Information Processing Systems conference in December, a premier conference in Machine Learning.
EDUCATIONAL ADVANCES (If applicable)