Science of Human Circumvention of Security - January 2017

Public Audience
Purpose: To highlight project progress. Information is generally at a higher level which is accessible to the interested public. All information contained in the report (regions 1-3) is a Government Deliverable/CDRL.

PI(s): Tao Xie

Co-PI(s): Jim Blythe (USC), Ross Koppel (UPenn), and Sean Smith (Dartmouth)

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released November 2012.

Our project most closely aligns with problem 5, "Understanding and Accounting for Human Behavior." However, it also pertains to problems 1, 2, and 3:

• Scalability and Composability: We want to understand not just the drivers of individual incidents of human circumvention, but also the net effect of these incidents.Included here are measures of the environment (physical, organizational, hierarchical, embeddedness within larger systems.)
• Policy-Governed Secure Collaboration: In order to create policies that in reality actually enable secure collaboration among users in varying domains, we need to understand and predict the de facto consequences of policies, not just the de juro ones.
• Security-Metrics-Driven Evaluation, Design, Development, and Deployment:Making sane decisions about what security controls to deploy requires understanding the de facto consequences of these deployments---instead of just pretending that circumvention by honest users never happens.

PUBLICATIONS
Papers published in this quarter as a result of this research. Include title, author(s), venue published/presented, and a short description or abstract. Identify which hard problem(s) the publication addressed. Papers that have not yet been published should be reported in region 2 below.

• J Walker and R Koppel, "For Healthcare Cybersecurity the Whole is Weaker than the Sum of the Parts," The Health Care Blog (THCB), September 23, 2016. (Relevant to HP5,1,2,3)
• X. Zeng, D. Li, W. Zheng, F. Xia, Y. Deng, W. Lam, W. Yang, T. Xie, "Automated Test Input Generation for Android: Are We Really There Yet in an Industrial Case?", 24th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE 2016), Industrial Track, Seattle, WA, November 13-18, 2016. (Relevant to HP5,1,2,3)
• P. McCauley, B. Nsiah-Ababio, J. Reed, F. Isiaka, T. Xie, "Preliminary Analysis of Code Hunt Data Set from a Contest", 2nd International Code Hunt Workshop on Educational Software Engineering (CHESE 2016), Seattle, WA, November 14, 2016. (Relevant to HP5,1,2,3)
• R. Koppel, V. Kothari, S.W. Smith, J. Blythe, "Beyond Pleading With or Restricting Users to Achieve Cyber Security Goals: Approaches to Understanding and Responding to Circumvention." (Position paper), CRA CCC Sociotechnical Cybersecurity Workshop, Hyattsville, MD, December 12-13, 2016. (Relevant to HP5,1,2,3)
• S.W. Smith, V. Kothari, J. Blythe, R. Koppel, "Flawed Mental Models Lead to Bad Cyber Security Decisions: Let's Do a Better Job." (Position paper), CRA CCC Sociotechnical Cybersecurity Workshop, Hyattsville, MD, December 12-13, 2016. (Relevant to HP5,1,2,3)

ACCOMPLISHMENT HIGHLIGHTS

Our goal is to improve aggregate security in light of rampant user circumvention of security policies and recommended security practices. We combine our unique expertise to tackle this problem of human circumvention of security using various approaches, including, but not limited to, semiotic modeling, surveys, behavioral experiments, and agent-based simulation. We seek to (a) enlighten security practitioners as to what users think and do, (b) bridge disconnects between security practitioners' mental models and reality, (c) develop tools to aid in security decisions, and (d) suggest better security solutions.

Via fieldwork in real-world enterprises, we have been identifying and cataloging types and causes of circumvention by well-intentioned users. We are using help desk logs, records security-related computer changes, analysis of user behavior in situ, and surveys--in addition to interviews and observations. We then began to build and validate models of usage and circumvention behavior, for individuals and then for populations within an enterprise--as well as developing some typologies of the deeper patterns and causes. For example, we've adapted previous work in the area of semiotics to build a model to capture mismorphisms, disconnects between various actors' mental models, the system's representation of reality, and the reality itself. We believe improvements to this model may enable us to meaningfully classify hosts of security issues and suggest methods to address them.

We have been developing questionnaires for both high-level computer security professionals and general users. These results will improve our understanding of perceptions, attitudes, and behaviors of both security practitioners and general users. Indeed, results may improve security practitioners' decisions directly or indirectly by providing requisite data to build faithful models of human behavior that can inform security practitioners. We have conducted surveys on a small scale and have done initial analysis of results. We are now conducting surveys on a larger scale.

Using DASH, an agent-based modeling platform, we have built and are continually improving upon a password simulation for measuring the security provided by a password composition policy, taking into account human circumventions such as writing down and reusing passwords. We continue to refine the model to improve its faithfulness to reality and usefulness.

We're building a platform to conduct password security experiments on Mechanical Turk that will provide data on why, how, and when users circumvent recommended password practices. We are completing final stages of testing and aim to perform these experiments in the near future.

We are collaborating with researchers at University of Pennsylvania who specialize in simulating and checking Markov chain models. We are exploring ways to blend these Markov-based models with our DASH model to tackle security problems using ground-truth data from the Mechanical Turk experiments and the literature.

We have been continually developing a platform, called DASH, for agent-based simulations of circumventive behavior in order to understand their causes and consequences. We have largely completed the re-implementation of DASH in Python and have built several agents on the new platform, including models for password behavior, authentication on shared computers and attackers.

We now list some accomplishment highlights from the latest quarter. (Section 2 below has a more complete presentation.)

• PIs Smith, Blythe and Koppel attended the NSA-led multi-project meeting in Maryland. Each spoke about their work.
• PI Koppel distributed the questionnaires on cybersecurity to the project leaders following the meeting.
• PIs Smith and Koppel attended the Sociotechnical Cybersecurity Workshop (see presentations above).
• PIs Blythe, Koppel, and Smith, and Dartmouth graduate student Kothari submitted a paper entitled "Password Logbooks and User Reviews: What Amazon Tells Us About Usable Security" to USEC 2017.
• We have also generally been making progress on all fronts. In addition to above, we've been working toward developing a DASH-based simulation that explores how emotion affects user susceptibility to phishing. We're developing a DASH-based simulation that focuses on modeling resource contention in a medical setting wherein clinicians must share computers. And we are nearing completion of testing the Mechanical Turk password experiment.
• PI Xie and collaborators co-authored a short paper titled "Automated Test Input Generation for Android: Are We Really There Yet in an Industrial Case?" presented in FSE 2016 Industrial Track.
• PI Xie and collaborating undergraduate students co-authored a workshop paper titled "Preliminary Analysis of Code Hunt Data Set from a Contest" presented in CHESE 2016.
• Supported graduate student Dengfeng Li supervised by PI Xie presented a student poster on "User-Centric Mobile Security Assessment" at the 2016 Fall Science of Security Quarterly Meeting.
• Supported graduate student Dengfeng Li supervised by PI Xie presented a UIUC Joint TSS/SoS Seminar talk on "Towards Privacy-Preserving Mobile Utility Apps: A Balancing Act" at UIUC CSL Auditorium (B02) on November 29 2016.