Model-Based Explanation For Human-in-the-Loop Security - January 2020
PI(s), Co-PI(s), Researchers: David Garlan, Bradley Schmerl (CMU)
HARD PROBLEM(S) ADDRESSED
Human Behavior
Metrics
Resilient Architectures
We are addressing human behavior by providing understandable explanations for automated mitigation plans generated by self-protecting systems that use various models of the software, network, and attack. We are addressing resilience by providing defense plans that are automatically generated as the system runs and accounting for current context, system state, observable properties of the attacker, and potential observable operations of the defense.
PUBLICATIONS
None.
PUBLIC ACCOMPLISHMENT HIGHLIGHTS
End-users of planning agents require trust that the agents make decisions in ways that conform to what the users want. In many real-world applications of planning, multiple optimization objectives, including security, are often involved. Thus, planning agents' decisions can involve complex tradeoffs among competing objectives. It can be difficult for an end-user to understand why an agent decides on a particular planning solution on the basis of its objective values, particulary when there is a potential misalignment between the agent and the user's preference for the different planning objectives. As a result, the user may not know whether the agent's decision is the best option with respect to their own values and preference. In this work, we contribute an explainable planning approach, based on contrastive explanation, that enables the agent to communicate its preference for the different planning objectives and help the user better understand whether the agent's decision is optimal with respect to their own preference, despite a potential value misalignment. We conducted a human-subject experiment to evaluate the effectiveness of our explanation approach in the mobile robot navigation domain. The results show that our approach significantly improves the
users' ability and reliable confidence in determining whether the agent's decisions are in line with their preferences.
COMMUNITY ENGAGEMENTS (If applicable)
EDUCATIONAL ADVANCES (If applicable)