Data-Driven Model-Based Decision-Making - April 2015

Public Audience
Purpose: To highlight project progress. Information is generally at a higher level which is accessible to the interested public. All information contained in the report (regions 1-3) is a Government Deliverable/CDRL.

PIs: William Sanders, Masooda Bashir, David Nicol, and Aad Van Moorsel

Researchers: Ken Keefe, Mohamad Moureddine, Charles Morriset, and Rob Cain

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released November 2012.

• Predictive Security Metrics - System security analysis requires a holistic approach that considers the behavior of non-human subsystem, bad actors or adversaries, and expected human participants such as users and system administrators. We are developing the HITOP modeling formalism to formally describe the behavior of human participants and how their decisions affect overall system performance and security. With this modeling methodology and the tool support we are developing, we will produce quantitative security metrics for cyber-human systems.
• Human Behavior - Modeling and evaluating human behavior is challenging, but it is an imperative component in security analysis. Stochastic modeling serves as a good approximation of human behavior, but we intend to do more with the HITOP method, which considers a task based process modeling language that evaluates a human's opportunity, willingness, and capability to perform individual tasks in their daily behavior. Partnered with an effective data collection strategy to validate model parameters, we are working to provide a sound model of human behavior.

PUBLICATIONS
Papers published in this quarter as a result of this research. Include title, author(s), venue published/presented, and a short description or abstract. Identify which hard problem(s) the publication addressed. Papers that have not yet been published should be reported in region 2 below.

See pending publications sections.

ACCOMPLISHMENT HIGHLIGHTS

This quarter, we identified several variables affecting the human behavior when it comes to making security decisions. We identified these variables by examining the interviews reported in publications from the usable security field and identifying the most common factors mentioned by the participants. We then categorized these variables into three categories depending on whether they are influenced by the business environment, certain cognitive factors, or personal factors. Finally, we surveyed popular human models from the fields of artificial intelligence and behavioral economics, in order to provide motivation for the models of human behavior we will develop to characterize the variables that we identified.

We also completed a prototype implementation of the HITOP modeling formalism in the Mobius Framework. We are using this implementation to develop a realistic case study to evaluate the Human-Influenced Task-Oriented Process (HITOP) formalism. The case study simulates a typical engineering firm trying to protect its important assets, such as employee and customer information and unpublished designs. We plan to model the human users and system administrators using the HITOP formalism, the firm's network and systems using stochastic activity networks (SANs).

We also made progress on developing an approach to collect experimental security data to populate models. In particular, based on the literature review carried out in the previous quarter with respect to data collection strategies, we developed a mathematical optimization formulation for the problem of optimizing data collection strategies. The optimization problem is an integer optimization problem, investing units of data collection effort in various parameters of the model. The allocation of data collection efforts then minimizes the remaining variance of the model outcome, thus assuring that data has been collected for the most important model parameters.