Science of Human Circumvention of Security - July 2015

Public Audience
Purpose: To highlight project progress. Information is generally at a higher level which is accessible to the interested public. All information contained in the report (regions 1-3) is a Government Deliverable/CDRL.

PI(s): Tao Xie

Co-PI(s): Jim Blythe (USC), Ross Koppel (UPenn), and Sean Smith (Dartmouth)

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released November 2012.

Our project most closely aligns with problem 5, "Understanding and Accounting for Human Behavior." However, it also pertains to problems 1, 2, and 3:

• Scalability and Composability: We want to understand not just the drivers of individual incidents of human circumvention, but also the net effect of these incidents.Included here are measures of the environment (physical, organizational, hierarchical, embeddedness within larger systems.)
• Policy-Governed Secure Collaboration: In order to create policies that in reality actually enable secure collaboration among users in varying domains, we need to understand and predict the de facto consequences of policies, not just the de juro ones.
• Security-Metrics-Driven Evaluation, Design, Development, and Deployment:Making sane decisions about what security controls to deploy requires understanding the de facto consequences of these deployments---instead of just pretending that circumvention by honest users never happens.

PUBLICATIONS
Papers published in this quarter as a result of this research. Include title, author(s), venue published/presented, and a short description or abstract. Identify which hard problem(s) the publication addressed. Papers that have not yet been published should be reported in region 2 below.

[1] S.W. Smith, R. Koppel, J. Blythe, V. Kothari, "Mismorphism: a Semiotic Model of Computer Security Circumvention," invited talk, 9th International Symposium on Human Aspects of Information Security and Assurance, July 2015.

Addresses Problems 5,1,2,3

[2] V. Kothari, J. Blythe, S.W. Smith, R. Koppel. "Measuring the Security Impacts of Password Policies Using Cognitive Behavioral Agent-Based Modeling.," Symposium and Bootcamp on the Science of Security (HotSoS 2015). ACM. April 2015.

Abstract: Agent-based modeling can serve as a valuable asset to security personnel who wish to better understand the security landscape within their organization, especially as it relates to user behavior and circumvention. In this paper, we argue in favor of cognitive behavioral agent-based modeling for usable security and report on our work on developing an agent-based model for a password management scenario. We perform a number of trials and a sensitivity analysis that provide valuable insights into improving security (e.g., an organization that wishes to suppress one form of circumvention may want to endorse another form of circumvention).

This publication addresses Problems 5,1,2,3.

[3] S.W. Smith, R. Koppel, J. Blythe, V. Kothari. "Mismorphism: A Semiotic Model of Computer Security Circumvention (Poster Abstract)." Symposium and Bootcamp on the Science of Security (HotSoS 2015). ACM. April 2015.

[4] W. Yang, X. Xiao, B Andow, S. Li, T. Xie, and W. Enck. "AppContext: Differentiating Malicious and Benign Mobile App Behavior Under Contexts." In Proceedings of the 37th International Conference on Software Engineering (ICSE 2015), Florence, Italy, May 2015.

Abstract: Mobile malware attempts to evade detection during app analysis by mimicking security-sensitive behaviors of benign apps that provide similar functionality (e.g., sending SMS messages), and suppressing their payload to reduce the chance of being observed (e.g., executing only its payload at night). Since current approaches focus their analyses on the types of security-sensitive resources being accessed (e.g., network), these evasive techniques in malware make differentiating between malicious and benign app behaviors a difficult task during app analysis. We propose that the malicious and benign behaviors within apps can be differentiated based on the contexts that trigger security-sensitive behaviors, i.e., the events and conditions that cause the security-sensitive behaviors to occur. In this work, we introduce AppContext, an approach of static program analysis that extracts the contexts of security-sensitive behaviors to assist app analysis in differentiating between malicious and benign behaviors. We implement a prototype of AppContext and evaluate AppContext on 202 malicious apps from various malware datasets, and 633 benign apps from the Google Play Store. AppContext correctly identifies 192 malicious apps with 87.7% precision and 95% recall. Our evaluation results suggest that the maliciousness of a security-sensitive behavior is more closely related to the intention of the behavior (reflected via contexts) than the type of the security-sensitive resources that the behavior accesses.

This paper addresses Problems 5,1,3.

[5] T. Xie, J. Bishop, N. Tillmann, and J. de Halleux. "Gamifying Software Security Education and Training via Secure Coding Duels in Code Hunt". In Proceedings of Symposium and Bootcamp on the Science of Security (HotSoS 2015), Urbana, IL, April 2015.

Abstract: Sophistication and flexibility of software development make it easy to leave security vulnerabilities in software applications for attackers. It is critical to educate and train software engineers to avoid introducing vulnerabilities in software applications in the first place such as adopting secure coding mechanisms and conducting security testing. A number of websites provide training grounds to train people's hacking skills, which are highly related to security testing skills, and train people's secure coding skills. However, there exists no interactive gaming platform for instilling gaming aspects into the education and training of secure coding. To address this issue, we propose to construct secure coding duels in Code Hunt, a high-impact serious gaming platform released by Microsoft Research. In Code Hunt, a coding duel consists of two code segments: a secret code segment and a player-visible code segment. To solve a coding duel, a player iteratively modifies the player-visible code segment to match the functional behaviors of the secret code segment. During the duel-solving process, the player is given clues as a set of automatically generated test cases to characterize sample functional behaviors of the secret code segment. The game aspect in Code Hunt is to recognize a pattern from the test cases, and to re-engineer the player-visible code segment to exhibit the expected behaviors. Secure coding duels proposed in this work are coding duels that are carefully designed to train players' secure coding skills, such as sufficient input validation and access control.

This paper addresses Problems 5,1,3.

Xie presented this in April 2015.

ACCOMPLISHMENT HIGHLIGHTS

Via fieldwork in real-world enterprises, we have been identifying and cataloging types and causes of circumvention by well-intentioned users. We are using help desk logs, records security-related computer changes, analysis of user behavior in situ, and surveys--in addition to interviews and observations. We then began to build and validate models of usage and circumvention behavior, for individuals and then for populations within an enterprise--as well as developing some typologies of the deeper patterns and causes.