Biblio

Probabilistic model checking is a useful technique for specifying and verifying properties of stochastic systems including randomized protocols and reinforcement learning models. However, these methods rely on the assumed structure and probabilities of certain system transitions. These assumptions may be incorrect, and may even be violated by an adversary who gains control of some system components. In this paper, we develop a formal framework for adversarial robustness in systems modeled as discrete time Markov chains (DTMCs). We base our framework on existing methods for verifying probabilistic temporal logic properties and extend it to include deterministic, memoryless policies acting in Markov decision processes (MDPs). Our framework includes a flexible approach for specifying structure-preserving and non structure-preserving adversarial models. We outline a class of threat models under which adversaries can perturb system transitions, constrained by an ε ball around the original transition probabilities. We define three main DTMC adversarial robustness problems: adversarial robustness verification, maximal δ synthesis, and worst case attack synthesis. We present two optimization-based solutions to these three problems, leveraging traditional and parametric probabilistic model checking techniques. We then evaluate our solutions on two stochastic protocols and a collection of Grid World case studies, which model an agent acting in an environment described as an MDP. We find that the parametric solution results in fast computation for small parameter spaces. In the case of less restrictive (stronger) adversaries, the number of parameters increases, and directly computing property satisfaction probabilities is more scalable. We demonstrate the usefulness of our definitions and solutions by comparing system outcomes over various properties, threat models, and case studies.

Kernel embeddings of distributions have recently gained significant attention in the machine learning community as a data-driven technique for representing probability distributions. Broadly, these techniques enable efficient computation of expectations by representing integral operators as elements in a reproducing kernel Hilbert space. We apply these techniques to the area of stochastic optimal control theory and present a method to compute approximately optimal policies for stochastic systems with arbitrary disturbances. Our approach reduces the optimization problem to a linear program, which can easily be solved via the Lagrangian dual, without resorting to gradient-based optimization algorithms. We focus on discrete- time dynamic programming, and demonstrate our proposed approach on a linear regulation problem, and on a nonlinear target tracking problem. This approach is broadly applicable to a wide variety of optimal control problems, and provides a means of working with stochastic systems in a data-driven setting.

This work investigates the problem of analyzing privacy of abrupt changes for general Markov processes. These processes may be affected by changes, or exogenous signals, that need to remain private. Privacy refers to the disclosure of information of these changes through observations of the underlying Markov chain. In contrast to previous work on privacy, we study the problem for an online sequence of data. We use theoretical tools from optimal detection theory to motivate a definition of online privacy based on the average amount of information per observation of the stochastic system in consideration. Two cases are considered: the full-information case, where the eavesdropper measures all but the signals that indicate a change, and the limited-information case, where the eavesdropper only measures the state of the Markov process. For both cases, we provide ways to derive privacy upper-bounds and compute policies that attain a higher privacy level. It turns out that the problem of computing privacy-aware policies is concave, and we conclude with some examples and numerical simulations for both cases.

The output feedback problem of finite-time command filtering for nonlinear systems with random disturbance is addressed in this paper. This is the first time that command filtering and output feedback are integrated so that a nonlinear system with random disturbance converge rapidly in finite time. The uncertain functions and unmeasured states are estimated by the fuzzy logic system (FLS) and nonlinear state observer, respectively. Based on the adaptive framework, command filtering technology is applied to mitigate the problem of ``term explosion'' inherent in traditional methods, and error compensation mechanism is considered to improve the control performance of the system. The developed output feedback controller ensures the boundedness of all signals in the stochastic system within a finite time, and the convergence residual can converge to a small region. The validity of this scheme is well verified in a numerical example.

This paper investigates the event-triggered control (ETC) problem for stochastic networked control systems (NCSs) with exogenous disturbances and Denial-of-Service (DoS) attacks. The ETC strategy is proposed to reduce the utilization of network resource while defending the DoS attacks. Based on the introduced ETC strategy, sufficient conditions, which rely on the frequency and duration properties of DoS attacks, are obtained to achieve the stochastic input-to-state stability and Zeno-freeness of the ETC stochastic NCSs. An example of air vehicle system is given to explain the effectiveness of proposed ETC strategy.

Infrastructural systems such as the electricity grid, healthcare, and transportation networks today rely increasingly on the joint functioning of networked information systems and physical components, in short, on cyber-physical architectures. Despite tremendous advances in cryptography, physical-layer security and authentication, information attacks, both passive such as eavesdropping, and active such as unauthorized data injection, continue to thwart the reliable functioning of networked systems. In systems with joint cyber-physical functionality, the ability of an adversary to monitor transmitted information or introduce false information can lead to sensitive user data being leaked or result in critical damages to the underlying physical system. This paper investigates two broad challenges in information security in cyber-physical systems (CPSs): preventing retrieval of internal physical system information through monitored external cyber flows, and limiting the modification of physical system functioning through compromised cyber flows. A rigorous analytical framework grounded on information-theoretic security is developed to study these challenges in a general stochastic control system abstraction-a theoretical building block for CPSs-with the objectives of quantifying the fundamental tradeoffs between information security and physical system performance, and through the process, designing provably secure controller policies. Recent results are presented that establish the theoretical basis for the framework, in addition to practical applications in timing analysis of anonymous systems, and demand response systems in a smart electricity grid.

In this paper, we propose a robust Nash strategy for a class of uncertain Markov jump delay stochastic systems (UMJDSSs) via static output feedback (SOF). After establishing the extended bounded real lemma for UMJDSS, the conditions for the existence of a robust Nash strategy set are determined by means of cross coupled stochastic matrix inequalities (CCSMIs). In order to solve the SOF problem, an heuristic algorithm is developed based on the algebraic equations and the linear matrix inequalities (LMIs). In particular, it is shown that robust convergence is guaranteed under a new convergence condition. Finally, a practical numerical example based on the congestion control for active queue management is provided to demonstrate the reliability and usefulness of the proposed design scheme.

The security of Cyber-physical systems has been a hot topic in recent years. There are two main focuses in this area: Firstly, what kind of attacks can avoid detection, i.e., the stealthiness of attacks. Secondly, what kind of systems can stay stable under stealthy attacks, i.e., the invulnerability of systems. In this paper, we will give a detailed characterization for stealthy attacks and detection criterion for such attacks. We will also study conditions for the vulnerability of a stochastic linear system under stealthy attacks.

This paper is concerned with the stabilization problem of cyber physical system (CPS) exposed to a random replay attack. The study will ignore the effects of communication delays and packet losses, and the attention will be focused on the effect of replay attack on the stability of (CPS). The closed-loop system is modeled as Markovian jump linear system with two jumping parameters. Linear matrix inequality (LMI) formulation is used to give a condition for stochastic stabilization of the system. Finally the theory is illustrated through a numerical example.

Mitigating the effect of Distributed Denial of Service (DDoS) attacks in wired/wireless networks is a problem of extreme importance. The present paper investigates this problem and proposes a secure AQM to encounter the effects of DDoS attacks on queue's router. The employed method relies on modelling the TCP/AQM system subjected to different DoS attack rate where the resulting closed-loop system is expressed as new Markovian Jump Linear System (MJLS). Sufficient delay-dependent conditions which guarantee the syntheses of a stabilizing control for the closed-loop system with a guaranteed cost J* are derived. Finally, a numerical example is displayed.

We introduce a new defense mechanism for stochastic control systems with control objectives, to enhance their resilience before the detection of any attacks. To this end, we cautiously design the outputs of the sensors that monitor the state of the system since the attackers need the sensor outputs for their malicious objectives in stochastic control scenarios. Different from the defense mechanisms that seek to detect infiltration or to improve detectability of the attacks, the proposed approach seeks to minimize the damage of possible attacks before they actually have even been detected. We, specifically, consider a controlled Gauss-Markov process, where the controller could have been infiltrated into at any time within the system's operation. Within the framework of game-theoretic hierarchical equilibrium, we provide a semi-definite programming based algorithm to compute the optimal linear secure sensor outputs that enhance the resiliency of control systems prior to attack detection.

This paper deals with the robust H∞ cyber-attacks estimation problem for control systems under stochastic cyber-attacks and disturbances. The focus is on designing a H∞ filter which maximize the attack sensitivity and minimize the effect of disturbances. The design requires not only the disturbance attenuation, but also the residual to remain the attack sensitivity as much as possible while the effect of disturbance is minimized. A stochastic model of control system with stochastic cyber-attacks which satisfy the Markovian stochastic process is constructed. And we also present the stochastic attack models that a control system is possibly exposed to. Furthermore, applying H∞ filtering technique-based on linear matrix inequalities (LMIs), the paper obtains sufficient conditions that ensure the filtering error dynamic is asymptotically stable and satisfies a prescribed ratio between cyber-attack sensitivity and disturbance sensitivity. Finally, the results are applied to the control of a Quadruple-tank process (QTP) under a stochastic cyber-attack and a stochastic disturbance. The simulation results underline that the designed filters is effective and feasible in practical application.