Visible to the public Securing Safety-Critical Machine Learning Algorithms - January 2020Conflict Detection Enabled

Submitted by Jamie Presken on Mon, 12/09/2019 - 1:07pm

PI(s), Co-PI(s), Researchers: Lujo Bauer, Matt Fredrikson (CMU), Mike Reiter (UNC)

HARD PROBLEM(S) ADDRESSED

This project addresses the following hard problems: developing security metrics and developing resilient architectures. Both problems are tackled in the context of deep neural networks, which are a particularly popular and performant type of machine learning algorithm. This project develops metrics that characterize the degree to which a neural-network-based classifier can be evaded through practically realizable, inconspicuous attacks. The project also develops architectures for neural networks that would make them robust to adversarial examples.

PUBLICATIONS

  • Mahmood Sharif, Lujo Bauer, and Michael K. Reiter. n-ML: Mitigating adversarial examples via ensembles of topologically manipulated classifiers. arXiv preprint 1912.09059, December 2019.

  • Mahmood Sharif, Lucas Keane, Lujo Bauer, Michael K. Reiter, and Saurabh Shintre. Optimization-guided binary diversification to mislead neural networks for malware detection. arXiv preprint 1912.09064, December 2019.

PUBLIC ACCOMPLISHMENT HIGHLIGHTS

  • We developed a new approach to train ensembles of classifiers to better resist attempts to create malicious inputs that would be misclassified. Similar to n-version programming, this approach relies on the assumption that each classifier will make mistakes independently of the others. This assumption typically does not hold for multiple versions of an ML classifier, which are prone to making the same mistakes, and the innovation in our work was in the method for training classifiers to be more purposefully diverse, particularly under adversarial conditions.
  • We have also continued our study of network pruning techniques to enhance robustness. Our approach is based on attribution measurements of internal neurons, and aims to identify features that are pivotal for adversarial examples but not necessary for correct classification of normal inputs. Our experiments to date suggest that it is possible to identify and remove such non-robust features for norm-bounded attacks, but suggest that physical attacks may rely on different sets of features that cannot be pruned without significant impact on model performance.

COMMUNITY ENGAGEMENTS (If applicable)

Bauer presented work that was part of this project at a seminar in CMU-Africa in Kigali, Rwanda; as part of a keynote for the German American Chambers of Commerce East Coast Industry Forum's meeting in Pittsburgh; a seminar at ETH Zurich; and at the CPS Verification & Validation Workshop at CMU.

EDUCATIONAL ADVANCES (If applicable)

N/A this quarter

Groups: