Lablet Research: Human Behavior & Cybersecurity

EXECUTIVE SUMMARY:

Over the past year the, NSA Science of Security lablets engaged in 12 NSA-approved research projects addressing the hard problem of Human Behavior and Cybersecurity. Both CMU and UIUC worked with non-lablet universities on SoS research, effectively expanding the SoS community. In addition to the lablets, other universities involved in SoS research include Berkeley, U Pitt, UTSA, University of Newcastle, USC, UPenn, and Dartmouth. Several of the projects addressed other hard problems, most frequently Security-Metrics-Driven Evaluation, Design, Development, and Deployment. The projects are in various stages of maturity, and several have led to publications and/or conference presentations. Summaries of the projects, highlights, and publications are presented below.

1. USE: User Security Behavior (CMU/Berkeley/University of Pittsburgh Collaborative Proposal)

SUMMARY: The Security Behavior Observatory addresses the hard problem of "Understanding and Accounting for Human Behavior" by collecting data directly from people's own home computers, thereby capturing people's computing behavior "in the wild". This data is the closest to the ground truth of the users' everyday security and privacy challenges that the research community has ever collected. We expect the insights discovered by analyzing this data will profoundly impact multiple research domains, including but not limited to behavioral sciences, computer security & privacy, economics, and human-computer interaction. By its very nature - building infrastructure to collect data, then collecting, and eventually analyzing the data - the project has a long set up phase. As a result, it will likely be much more publication-centered toward the second half of its projected duration. However, we are confident that the greater number and quality of sensors we are building, and the more secure, reliable, and robust infrastructure we continue to build will provide more and better data, resulting in more and stronger publications. However, now that we are launching our data collection pilot study, we hope to compile the lessons learnt about building and launching such a large-scale field study into an early publication. We also hope the pilot will go smoothly enough that we could submit a paper with early results from the short-term data collected. (ID#:14-3330)

HIGHLIGHTS and PUBLICATIONS

• We have launched our data collection architecture pilot study, and have thus far not encountered any technical challenges.
• With the launch of our pilot study, we are now also pilot testing numerous data collection sensors, which are collecting live field data on client machines' processes, filesystem meta-data (e.g., file path, file size, date created, date modified, permissions), network packet headers, Windows security logs, Windows updates, installed software, and wireless access points.
• We have published the following technical report describing our data collection architecture and the various issues and design decisions surrounding building and deploying a large-scale data collection infrastructure: A. Forget, S. Komanduri, A. Acquisti, N. Christin, L.F. Cranor, R. Telang. "Security Behavior Observatory: Infrastructure for Long-term Monitoring of Client Machines." Carnegie Mellon University CyLab Technical Report CMU-CyLab-14-009. https://www.cylab.cmu.edu/research/techreports/2014/tr_cylab14009.html (accessed 2014-09-05)
• We have also given an invited presentation of our project, as well as an archival poster presentation, at the IEEE Symposium and Bootcamp on the Science of Security 2014 (HotSoS, http://www.csc2.ncsu.edu/conferences/hotsos/index.html).

2. Usable Formal Methods for the Design and Composition of Security Privacy Policies (CMU/UTSA Collaborative Proposal)

SUMMARY: Our research is based on theories in psychology concerning how designers comprehend and interpret their environment, how they plan and project solutions into the future, with the aim of better understanding how these activities exist in designing more secure systems. These are not typical models of attackers and defenders, but models of developer behavior, including our ability to influence that behavior with interventions. The project also addresses the hard problem of Security-Metrics-Driven-Evaluation, Design, Development and Deployment. (ID#:14-3331)

HIGHLIGHTS and PUBLICATIONS:

• We developed a repository and search tool that security analysts can use to select from 176 security patterns that were mined from a total of 21 different publications.
• We designed a survey protocol to collect security analyst risk perceptions for formalization in Fuzzy Logic. We plan to evaluate the formalization to check whether it can predict co-dependencies between security requirements as increasing or decreasing perceptions of security risk with respect to specific threat scenarios.
• Hui Shen, Ram Krishnan, Rocky Slavin, and Jianwei Niu. "Sequence Diagram Aided Privacy Policy Specification", revision submitted for publication: IEEE Transactions on Dependable and Secure Computing in August 2014.
• H. Hibshi, T. Breaux, M. Riaz, L. Williams. "Discovering Decision-Making Patterns for Security Novices and Experts", In Submission: International Journal of Secure Software Engineering, 2014.
• H. Hibshi, T. Breaux, M. Riaz, L. Williams. "A Framework to Measure Experts' Decision Making in Security Requirements Analysis," IEEE 1st International Workshop on Evolving Security and Privacy Requirements Engineering, pp. 13-18, 2014.
• R. Slavin, J.M. Lehker, J. Niu, T. Breaux. "Managing Security Requirement Patterns Using Feature Diagram Hierarchies," IEEE 22nd International Requirements Engineering Conference, pp. 193-202, 2014.
• Slankas, J., Riaz, M. King, J., Williams, L. "Discovering Security Requirements from Natural Language," IEEE 22nd International Requirements Engineering Conference, 2014.
• Rao, H. Hibshi, T. Breaux, J-M. Lehker, J. Niu, "Less is More? Investigating the Role of Examples in Security Studies using Analogical Transfer," 2014 Symposium and Bootcamp on the Science of Security (HotSoS), Article 7.
• H. Hibshi, R. Slavin, J. Niu, T. Breaux, "Rethinking Security Requirements in RE Research," University of Texas at San Antonio, Technical Report #CS-TR-2014-001, January, 2014
• Riaz, M., Breaux, T., Williams, L. "On the Design of Empirical Studies to Evaluate Software Patterns: A Survey," Revision submitted for consideration: Information and Software Technology, 2014
• Breaux, T., Hibshi, H., Rao, A., Lehker, J.-M. "Towards a Framework for Pattern Experimentation: Understanding empirical validity in requirements engineering patterns." IEEE 2nd Workshop on Requirements Engineering Patterns (RePa'12), Chicago, Illinois, Sep. 2012, pp. 41-47.
• Slavin, R., Shen, H., Niu, J., "Characterizations and Boundaries of Security Requirements Patterns," IEEE 2nd Workshop on Requirements Engineering Patterns (RePa'12), Chicago, Illinois, Sep. 2012, pp. 48-53.

3. Leveraging the Effects of Cognitive Function on Input Device Analytics to Improve Security (NCSU)

SUMMARY: Our work addresses understanding human behavior through observations of input device usage. The basic principles we are developing will enable new avenues for characterizing risk and identifying malicious (or accidental) uses of systems that lead to security problems. (ID#:14-3332)

HIGHLIGHTS and PUBLICATIONS:

• We have extensively tested a typing game, which has been under development for the past two quarters.
• Our early analysis of data from a pilot evaluation has been completed, and it resulted in a redesign which we expect will lead to much higher quality data.
• The final version is ready for deployment, and we have commenced data collection.
• The team has acquired an eye-tracking device and began developing software for it to be integrated into the experiment, for additional instrumentation. -
• Data analysis on the mouse movement patterns during the concentration game has also progressed, and we have identified a number of characteristic patterns in movement hesitations.

4. A Human Information-Processing Analysis of Online Deception Detection (NCSU)

SUMMARY: Predicting individual users' judgments and decisions regarding possible online deception. Our research addresses this problem within the context of examining user decisions with regard to phishing attacks. This work is grounded within the scientific literature on human decision-making processes. (ID#:14-3333)

HIGHLIGHTS and PUBLICATIONS

• Continued to modify the design of our Google Chrome browser extension to protect against phishing attacks, through iterative evaluation.
• Completed procedural details for the study, "Browser Extension to Prevent Phishing Attack", initially designed in the last quarter, including preparation of fliers for recruiting subjects, consent forms, questionnaires, and the interface mentioned in the first bullet, as well as the protocol for conducting the experiment.
• Submitted an application to the Institutional Review Board at Purdue University, which was approved.

5. Warning of Phishing Attacks: Supporting Human Information Processing, Identifying Phishing Deception Indicators, and Reducing Vulnerability (NCSU)

SUMMARY: This preliminary work in understanding how mental models vary between novice users, experts (such as IT professionals), and hackers should be useful in accomplishing the ultimate goal of the work: to build secure systems that reduce user vulnerability to phishing. (ID#:14-3334)

HIGHLIGHTS and PUBLICATIONS:

• We have completed data collection from the novices recruited for the mental models experiment. In preparation for the Industry Day Lablet meeting at NCSU on Oct. 24, we plan to recruit our "knowledgeable" sample of computer security professionals so that we can complete data collection on this project. By recruiting from these two diverse samples that vary considerably on security-related knowledge, we hope to expose how novices differ from experts on how they conceptualize system security attributes. Knowledge of these differences in mental models should allow us to recommend interventions that can promote security for all users (but most specifically novices).
• Preliminary data analysis on this project has been initiated.
• To demonstrate our knowledge dissemination, we are presenting our Lablet research at the Oct. 24 (Industry Day), a meeting of the Carolinas Chapter of the Human Factors and Ergonomics Society (HFES) on Oct. 23, and at the international conference for HFES in Chicago from Oct. 27-31.
• Zielinska, O., Tembe, R., Hong, K. W., Xe, G., Murphy-Hill, E. & Mayhorn, C. B. (2014). "One Phish, Two Phish, How to Avoid the Internet Phish: Analysis of Training Strategies to Detect Phishing Emails." Proceedings of the Human Factors and Ergonomics Society 56th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.

6. Data-Driven Model-Based Decision-Making (UIUC/University of Newcastle Collaborative Proposal)

SUMMARY: Modeling and evaluating human behavior is challenging, but it is an imperative component in security analysis. Stochastic modeling serves as a good approximation of human behavior, but we intend to do more with the HITOP method, which considers a task based process modeling language that evaluates a human's opportunity, willingness, and capability to perform individual tasks in their daily behavior. Partnered with an effective data collection strategy to validate model parameters, we are working to provide a sound model of human behavior. This project also addresses the hard problem of Predictive Security Metrics. (ID#:14-3335)

HIGHLIGHTS and PUBLICATIONS:

• Newcastle University has lined up their research team for their work on the project.
• Regular team meetings at UIUC have commenced and planning for improvements to the current HITOP prototype has been completed.
• Full team kick off with Newcastle University has been scheduled for the first week of October.

7. Science of Human Circumvention of Security (UIUC/USC/UPenn/Dartmouth Collaborative Proposal)

SUMMARY: Via fieldwork in real-world enterprises, we have been identifying and cataloging types and causes of circumvention by well-intentioned users. We are using help desk logs, records security-related computer changes, analysis of user behavior in situ, and surveys---in addition to interviews and observations. We then began to build and validate models of usage and circumvention behavior, for individuals and then for populations within an enterprise. This project also addresses three other hard problems: Scalability and Composability; Policy-Governed Secure Collaboration; and Security-Metrics-Driven Evaluation, Design, Development, and Deployment. (ID#:14-3336)

HIGHLIGHTS and PUBLICATIONS:

• The JAMIA paper by Smith and Koppel on usability problems with health IT (pre-SHUCS, but related) received another accolade, this time from the International Medical Informatics Association, which also named it one of best papers of 2014. We are updating that paper to include discoveries from our analysis of the workaround corpora above.
• J. Blythe, R. Koppel, V. Kothari, and S. Smith. "Ethnography of Computer Security Evasions in Healthcare Settings: Circumvention as the Norm". HealthTech' 14: Proceedings of the 2014 USENIX Summit on Health Information Technologies, August 2014.
• R. Koppel. "Software Loved by its Vendors and Disliked by 70% of its Users: Two Trillion Dollars of Healthcare Information Technology's Promises and Disappointments". HealthTech'14: Keynote talk at the 2014 USENIX Summit on Health Information Technologies, August 2014.
• R. Koppel. "Software Loved by its Vendors and Disliked by 70% of its Users: Two Trillion Dollars of Healthcare Information Technology's Promises and Disappointments". HealthTech'14: Keynote talk at the 2014 USENIX Summit on Health Information Technologies, August 2014.

8. Human Behavior and Cyber Vulnerabilities (UMD)

SUMMARY: When a vulnerability is exploited, software vendors often release patches fixing the vulnerability. However, our prior research has shown that some vulnerabilities continue to be exploited more than four years after their disclosure. Why? We posit that there are both technical and sociological reasons for this. On the technical side, it is unclear how quickly security patches are disseminated, and how long it takes to patch all the vulnerable hosts on the Internet. On the sociological side, users/administrators may decide to delay the deployment of security patches. Our goal in this task is to validate and quantify these explanations. Specifically, we seek to characterize the rate of vulnerability patching, and to determine the factors--both technical and sociological--that influence the rate of applying patches. This project also addresses the hard problem of Security-Metrics-Driven Evaluation, Design, Development, and Deployment. (ID#:14-3337)

HIGHLIGHTS and PUBLICATIONS:

• We conducted a study to determine how SSL certificates were reissued and revoked in response to a widespread vulnerability, Heartbleed, that enabled undetectable key compromise. We conducted large-scale measurements and developed new methodologies to determine how the most-popular 1 million web sites reacted to this vulnerability in terms of certificate management, and how this impacts security for clients that use those web sites.
• We found that the vast majority of vulnerable certificates have not been reissued; further, of those domains that reissued certificates in response to Heartbleed, 60% did not revoke their vulnerable certificates. If those certificates are not eventually revoked, 20% of them will remain valid (i.e., will not expire) for two or more years. The ramifications of this findings are alarming: users will remain potentially vulnerable to malicious third parties using stolen keys to masquerade as a compromised site for a long time to come. We analyzed these trends with vulnerable Extended Validation (EV) certificates as well, and found that, while such certificates were handled with better security practices, those certificates still remain largely not reissued (67%) and not revoked (88%) even weeks after the vulnerability was made public.
• Liang Zhang, David Choffnes, Tudor Dumitras, Dave Levin, Alan Mislove, Aaron Schulman, and Christo Wilson. Analysis of SSL Certificate Reissues and Revocations in the Wake of Heartbleed. In Proceedings of the ACM Internet Measurement Conference (IMC'14), Vancouver, Canada, Nov 2014.

9. Does the Presence of Honest Users Affect Intruders' Behavior? (UMD)

SUMMARY: The underlying premise that drives many existing cybersecurity efforts is that once an attacker has gained access to a computer system, the compromised system is no longer under the victim's control and all is lost. While we agree that efforts to secure computer systems should focus on preventing system infiltration, attention should also be given to the study of situational factors that might mitigate the potential damage caused by a successful breach. This research task applies "soft science" (sociology, psychology, and criminology) to better understand the effect of system configurations and situational stimuli on the progression and development of system break-ins. (ID#:14-3338)

10. Understanding Developers' Reasoning about Privacy and Security (UMD)

SUMMARY: Our goal is to discover, understand, and quantify challenges that developers face in writing secure and privacy preserving programs. Several research thrusts will enable this goal. Qualitative studies of developers will discover cultural and workplace dynamics that encourage or discourage privacy and security by design. Experiments with alternative design schemas will test how to facilitate adoption. (ID#:14-3339)

HIGHLIGHTS and PUBLICATIONS:

• We have continued interviews with mobile application developers focused on cultural and workplace dynamics, and these are expected to progress over the course of the coming academic year.
• We have implemented a simplified version of the Bubbles platform including the Bubbles trusted viewer and the centralized database server. The Bubbles trusted viewer resides in a user's Android device and provides other applications with a trusted platform service. With the Bubbles platform, a user groups various application data into a single Bubble based on its context. Then the user can share a Bubble only with the people he has selected at the time of Bubble creation. The Bubble platform prevents any malicious applications from sharing the user data with anyone who is not authorized by the data owner. We are preparing for a user study to measure developers' reasoning about privacy and security vis-a-vis our platform. We will measure how well non-security-expert undergraduate students understand Bubble platform's security model and how easily they can convert a non-secure Android application into a secure, Bubble-compatible version. For this, we have implemented a simple Android application where a user can write a text memo and store it in a local database. The students will be provided with the Bubbles trusted viewer, the centralized database server and the simple Android application and will be asked to implement missing parts necessary for the compatibility with Bubbles platform.
• Krontiris, I., Langheinrichz, M. & Shilton, K. (2014). Trust and Privacy in Mobile Experience Sharing - Future Challenges and Avenues for Research. IEEE Communications, August 2014.

11. Reasoning about Protocols with Human Participants (UMD)

SUMMARY: Our purpose is to rigorously derive security properties of network-security protocols involving human participants and physical objects, where the limited computational capabilities of human participants and the physical properties of the objects affect the security properties of the protocols.

We first consider the example problem of electronic voting. Human voters are not explicitly taken into account since it is (implicitly) assumed that each voter has access to a trusted computer while voting. In our work we do not make this assumption, because voters voting from home might have malware on their computers that could be used to throw an election.

Some more recent voting protocols have been designed for human participants voting from untrusted computers, some relying on paper or other physical objects to obtain security guarantees. However, the security properties of these protocols are not well understood. We need a well-developed model to reason about these properties. Such a model would incorporate a human's computational capabilities and the properties of the physical objects. The model would then be used to reason about, and prove security of, the integrity and privacy properties of remote voting protocols such as Remotegrity (used for absentee voting by the City of Takoma Park for its 2011 municipal election).

In the short term, this project will focus on the development of the model of humans and the use of physical obects such as paper, and on the security properties of remote voting protocol Remotegrity. In the longer term---in addition to the general problem of the voting protocol---there are other problems where it is important to consider the fact that all protocol participants are not computers. For example, when a human logs into a website to make a financial transaction (such as a bank website, or a retirement account, or an e-commerce site), the human uses an untrusted computer and hence cannot be expected to correctly encrypt or sign messages. Can one use the techniques developed for electronic voting to develop simple and more secure protocols using physical objects and paper while using the untrusted computer to make the transaction? Can one prove the security properties of the proposed protocols? (ID#:14-3340)

HIGHLIGHTS and PUBLICATIONS:

• In accomplishments to date, we have begun formally specifying two remote voting protocols: Remotegrity and Helios. The former uses paper, while the latter does not. The former appears to be ``more secure'' --- in particular, with the ability to resolve disputes between the voting system or voting computer, which might claim that it encrypted the vote correctly, and the voter, who might dispute this claim. This project is rigorously examining this difference between the two protocols.

12. User-Centered Design for Security (UMD)

SUMMARY: Our goal is to better understand human behavior within security systems and through that learn knowledge propose, design, and build better security systems. There are several research thrusts involved in meeting this challenge: Understanding, Measuring, and Applying User Perceptions of Security and Usability; Measuring Queuing Language in User Graphical Password Selection; Improving Password Memorability; and Improving Password Memorability. This project also addresses the hard problem of Security-Metrics-Driven Evaluation, Design, Development, and Deployment. (ID#:14-3341)

HIGHLIGHTS and PUBLICATIONS:

• We have developed and pilot tested an experiment for improving password memorability through a timed reminder service based in principles of cognitive psychology. We are testing whether users, when prompted to login on a schedule with increasingly distant time periods, will better remember their passwords for multiple sites. If our hypothesis is correct, this will be one way to leverage lessons of HCI, cognitive science, and psychology to improve security of systems through better understanding human behavior.

Note:

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.