Securing Safety-Critical Machine Learning Algorithms - July 2018

PI(s), Co-PI(s), Researchers: Lujo Bauer, Matt Fredrikson (CMU), Mike Reiter (UNC)

HARD PROBLEM(S) ADDRESSED

This project addresses the following hard problems: developing security metrics and developing resilient architectures. Both problems are tackled in the context of deep neural networks, which are a particularly popular and performant type of machine learning algorithm. This project develops metrics that characterize the degree to which a neural-network-based classifier can be evaded through practically realizable, inconspicuous attacks. The project also develops architectures for neural networks that would make them robust to adversarial examples.

PUBLICATIONS

Mahmood Sharif, Lujo Bauer, and Michael K. Reiter. On the suitability of Lp-norms for creating and preventing adversarial examples. In Proceedings of The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security (in conjunction with the 2018 IEEE Conference on Computer Vision and Pattern Recognition), June 2018. (c) IEEE

Hard problems addressed: security metrics (primarily); resilient architectures (indirectly)

PUBLIC ACCOMPLISHMENT HIGHLIGHTS

We presented a paper at the CV-COPS workshop (The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security) discussing the suitability of different Lp-norms as metrics for determining whether inputs to machine-learning algorithms are "close" or "far" from each other. An input is held to be "adversarial" when it appears very similar to (or indistinguishable from) a benign input, but is classified differently. In the literature, similarity is typically measured using Lp-norms: two inputs are considered to be similar if their distance according to an Lp-norm is smaller than some application-specific threshold. This paper shows that such a notion of similarity is neither necessary nor sufficient for at least several datasets commonly used in research on adversarial machine learning. The paper's goal in pointing this out is to steer research away from using seemingly fragile metrics for defining adversarial inputs and what it means for a system to be robust to them.

COMMUNITY ENGAGEMENTS (If applicable)

EDUCATIONAL ADVANCES (If applicable)