Science of Human Circumvention of Security - January 2015

Public Audience
Purpose: To highlight project progress. Information is generally at a higher level which is accessible to the interested public. All information contained in the report (regions 1-3) is a Government Deliverable/CDRL.

PI(s): Tao Xie

Co-PI(s): Jim Blythe (USC), Ross Koppel (UPenn), and Sean Smith (Dartmouth)

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released November 2012.

Our project most closely aligns with problem 5, "Understanding and Accounting for Human Behavior." However, it also pertains to problems 1, 2, and 3:

• "Scalability and Composability": We want to understand not just the drivers of individual incidents of human circumvention, but also the net effect of these incidents. Included here are measures of the environment (physical, organizational, hierarchical, embeddedness within larger systems.)
• "Policy-Governed Secure Collaboration." In order to create policies that in reality actually enable secure collaboration among users in varying domains, we need to understand and predict the de facto consequences of policies, not just the de juro ones.
• "Security-Metrics-Driven Evaluation, Design, Development, and Deployment." Making sane decisions about what security controls to deploy requires understanding the de facto consequences of these deployments---instead of just pretending that circumvention by honest users never happens.

PUBLICATIONS
Papers published in this quarter as a result of this research. Include title, author(s), venue published/presented, and a short description or abstract. Identify which hard problem(s) the publication addressed. Papers that have not yet been published should be reported in region 2 below.

[1] R. Koppel, S. Smith, J. Blythe, and V. Kothari, "Workarounds to Computer Access in Healthcare Organizations: You Want My Password or a Dead Patient?" in Information Technology and Communications in Health (ITCH), 2015, (to appear).

Abstract: Workarounds to computer access in healthcare are sufficiently common that they often go unnoticed. Clinicians focus on patient care, not cybersecurity. We argue and demonstrate that understanding workarounds to healthcare workers' computer access requires not only analyses of computer rules, but also interviews and observations with clinicians. In addition, we illustrate the value of shadowing clinicians and conducing focus groups to understand their motivations and tradeoffs for circumvention. Ethnographic investigation of the medical workplace emerges as a critical method of research because in the inevitable conflict between even well-intended people versus the machines, it's the people who are the more creative, flexible, and motivated. We conducted interviews and observations with hundreds of medical workers and with 19 cybersecurity experts, CIOs, CMIOs, CTO, and IT workers to obtain their perceptions of computer security. We also shadowed clinicians as they worked. We present dozens of ways workers ingeniously circumvent security rules. The clinicians we studied were not "black hat" hackers, but just professionals seeking to accomplish their work despite the security technologies and regulations.

This publication addresses Problems 5,1,2,3.

[2] S.W. Smith, "Circumvention: why do good people do bad things, and what can we do about it?", Colloquium at Rutgers University, Department of Electrical and Computer Engineering, December 2014.

Abstract: The field of computer security implicitly believes that it's possible to write down a correct policy of which subjects can do what to which objects, and that all good users will abide by this policy. However, in real-world enterprises, this is simply not the case: users systematically circumvent security controls (and other aspects of IT) in order to get their jobs done. This talk will survey such circumvention scenarios, explore their underlying structure, and consider some ways to improve the situation by reducing the gap between the de facto and the de jure.

This talk was given by PI Smith and addresses Problems 5 and 2.

[3] S. Smith, R. Koppel, J. Blythe, V. Kothari, "Mismorphism: the Semiotics of Computer Security Circumvention", Submitted for publication, December 2014.
Abstract: In real world domains, from healthcare to power to finance, we deploy computer systems intended to streamline and improve the activities of human agents in the corresponding noncyber worlds (such as curing patients, or running power grids and hedge funds). However, talking to actual users (instead of, say, computer security experts) reveals endemic circumvention of the computer-embedded rules. Good-intentioned users, trying to get their jobs done, systematically work around security and other controls embedded in their IT systems. Over the last several years, via interviews, observations, surveys, and literature searches, we have explored this problem and have been collecting and analyzing a corpus of hundreds of circumvention and unusability scenarios. This work reveals that semiotic triads--but with mappings that ail to preserve structure--provide a framework to illuminate, organize, and analyze this fundamental problem. In this paper, we use this semiotic triangle framework to illuminate the problem, build a typology of circumvention scenarios, and outline steps for future work.

This submission addresses Problems 5,1,2,3.

[4] J. Blythe, R. Koppel, V. Kothari, S. Smith, "The Computer Security Perils of Reuse", Submitted for publication, December 2014.

Abstract: When developing new systems and components, designers routinely reuse existing policies, technologies, and architectures--frequently with little or no changes. Standard software engineering practice advocates the reuse of reliable components. However, our findings reveal that careless reuse in a different or even similar domain can introduce failures and new challenges that subvert security goals and impede organizational objectives. In this paper, we enumerate and analyze examples of reuse in various settings. We examine the motivations for reuse including its advantages, its disadvantages, human biases, and the real and the false economies it provides. We also study the factors and conditions that affect the success of reuse to provide recommendations to security personnel.

This submission addresses Problems 5,1,2,3.

In the previous quarter, we had an abstract accepted to the 2015 IEEE International Symposium on Technologies for Homeland Security; however, we did not submit the full paper, as the experiments were not completed in time.

ACCOMPLISHMENT HIGHLIGHTS

Via fieldwork in real-world enterprises, we have been identifying and cataloging types and causes of circumvention by well-intentioned users. We are using help desk logs, records security-related computer changes, analysis of user behavior in situ, and surveys---in addition to interviews and observations. We then began to build and validate models of usage and circumvention behavior, for individuals and then for populations within an enterprise.