Biblio

We present an online framework for learning and updating security policies in dynamic IT environments. It includes three components: a digital twin of the target system, which continuously collects data and evaluates learned policies; a system identification process, which periodically estimates system models based on the collected data; and a policy learning process that is based on reinforcement learning. To evaluate our framework, we apply it to an intrusion prevention use case that involves a dynamic IT infrastructure. Our results demonstrate that the framework automatically adapts security policies to changes in the IT infrastructure and that it outperforms a state-of-the-art method.

Trust is a cognitive ability that can be dependent on behavioral consistency. In this paper, a partially observable Markov Decision Process (POMDP)-based computational robot-human trust model is proposed for hand-over tasks in human-robot collaborative contexts. The robot's trust in its human partner is evaluated based on the human behavior estimates and object detection during the hand-over task. The human-robot hand-over process is parameterized as a partially observable Markov Decision Process. The proposed approach is verified in real-world human-robot collaborative tasks. Results show that our approach can be successfully applied to human-robot hand-over tasks to achieve high efficiency, reduce redundant robot movements, and realize predictability and mutual understanding of the task.

We present a system for interactive examination of learned security policies. It allows a user to traverse episodes of Markov decision processes in a controlled manner and to track the actions triggered by security policies. Similar to a software debugger, a user can continue or or halt an episode at any time step and inspect parameters and probability distributions of interest. The system enables insight into the structure of a given policy and in the behavior of a policy in edge cases. We demonstrate the system with a network intrusion use case. We examine the evolution of an IT infrastructure’s state and the actions prescribed by security policies while an attack occurs. The policies for the demonstration have been obtained through a reinforcement learning approach that includes a simulation system where policies are incrementally learned and an emulation system that produces statistics that drive the simulation runs.

A Robot Operating System (ROS) plays a significant role in organizing industrial robots for manufacturing. With an increasing number of the robots, the operators integrate a ROS with networked communication to share the data. This cyber-physical nature exposes the ROS to cyber attacks. To this end, this paper proposes a cross-layer approach to achieve secure and resilient control of a ROS. In the physical layer, due to the delay caused by the security mechanism, we design a time-delay controller for the ROS agent. In the cyber layer, we define cyber states and use Markov Decision Process to evaluate the tradeoffs between physical and security performance. Due to the uncertainty of the cyber state, we extend the MDP to a Partially Observed Markov Decision Process (POMDP). We propose a threshold solution based on our theoretical results. Finally, we present numerical examples to evaluate the performance of the secure and resilient mechanism.

Emerging zero-day vulnerabilities in information and communications technology systems make cyber defenses very challenging. In particular, the defender faces uncertainties of; e.g., system states and the locations and the impacts of vulnerabilities. In this paper, we study the defense problem on a computer network that is modeled as a partially observable Markov decision process on a Bayesian attack graph. We propose online algorithms which allow the defender to identify effective defense policies when utility functions are unknown a priori. The algorithm performance is verified via numerical simulations based on real-world attacks.