Biblio

Resilience and antifragility under duress present significant challenges for autonomic and self-adaptive systems operating in contested environments. In such settings, the system has to continually plan ahead, accounting for either an adversary or an environment that may negate its actions or degrade its capabilities. This will involve projecting future states, as well as assessing recovery options, counter-measures, and progress towards system goals. For antifragile systems to be effective, we envision three self-* properties to be of key importance: self-exploration, self-learning and self-training. Systems should be able to efficiently self-explore – using adversarial search – the potential impact of the adversary’s attacks and compute the most resilient responses. The exploration can be assisted by prior knowledge of the adversary’s capabilities and attack strategies, which can be self-learned – using opponent modelling – from previous attacks and interactions. The system can self-train – using reinforcement learning – such that it evolves and improves itself as a result of being attacked. This paper discusses those visions and outlines their realisation in AWaRE, a cyber-resilient and self-adaptive multi-agent system.

This study addresses the coordination issue of multi-agent systems under complicated actuator faults and cyber attacks. Distributed fault-tolerant design is developed with the estimated and output neighboring information in decentralized estimation observer. Criteria of reaching the exponential coordination of multi-agent systems with cyber attacks is obtained with average dwelling time and chattering bound method. Simulations validate the efficiency of the anti-attack fault-tolerant design.

This paper studies the differential privacy-preserving problem of discrete-time multi-agent systems (MASs) with antagonistic information, where the connected signed graph is structurally balanced. First, we introduce the bipartite consensus definitions in the sense of mean square and almost sure, respectively. Second, some criteria for mean square and almost sure bipartite consensus are derived, where the eventualy value is related to the gauge matrix and agents’ initial states. Third, we design the ε-differential privacy algorithm and characterize the tradeoff between differential privacy and system performance. Finally, simulations validate the effectiveness of the proposed algorithm.

In this paper, we discuss the methodology of generalizing the optimal control law from learned component tasks to unlearned composite tasks on Multi-Agent Systems (MASs), by using the linearity composition principle of linearly solvable optimal control (LSOC) problems. The proposed approach achieves both the compositionality and optimality of control actions simultaneously within the cooperative MAS framework in both discrete and continuous-time in a sample-efficient manner, which reduces the burden of re-computation of the optimal control solutions for the new task on the MASs. We investigate the application of the proposed approach on the MAS with coordination between agents. The experiments show feasible results in investigated scenarios, including both discrete and continuous dynamical systems for task generalization without resampling.

The concepts of information flow security and refinement are known to have had a troubled relationship ever since the seminal work of McLean. In this work we study refinements that support changes in data representation and semantics, including the addition of state variables that may induce new observational power or side channels. We propose a new epistemic approach to ignorance-preserving refinement where an abstract model is used as a specification of a system's permitted information flows, that may include the declassification of secret information. The core idea is to require that refinement steps must not induce observer knowledge that is not already available in the abstract model. Our study is set in the context of a class of shared variable multiagent models similar to interpreted systems in epistemic logic. We demonstrate the expressiveness of our framework through a series of small examples and compare our approach to existing, stricter notions of information-flow secure refinement based on bisimulations and noninterference preservation. Interestingly, noninterference preservation is not supported “out of the box” in our setting, because refinement steps may introduce new secrets that are independent of secrets already present at abstract level. To support verification, we first introduce a “cube-shaped” unwinding condition related to conditions recently studied in the context of value-dependent noninterference, kernel verification, and secure compilation. A fundamental problem with ignorance-preserving refinement, caused by the support for general data and observation refinement, is that sequential composability is lost. We propose a solution based on relational pre-and postconditions and illustrate its use together with unwinding on the oblivious RAM construction of Chung and Pass.

Due to the shortage of resources and services, agents are often in competition with each other. Excessive competition will lead to a social dilemma. Under the viewpoint of breaking social dilemma, we present a novel trust-based logic framework called Trust Computation Logic (TCL) for measure method to find the best partners to collaborate and automatically verifying trust in Multi-Agent Systems (MASs). TCL starts from defining trust state in Multi-Agent Systems, which is based on contradistinction between behavior in trust behavior library and in observation. In particular, a set of reasoning postulates along with formal proofs were put forward to support our measure process. Moreover, we introduce symbolic model checking algorithms to formally and automatically verify the system. Finally, the trust measure method and reported experimental results were evaluated by using DeepMind’s Sequential Social Dilemma (SSD) multi-agent game-theoretic environments.

The papers in this special section focus on resilient control in large-scae networked cyber-physical systems. These papers deal with the opportunities offered by these emerging technologies to mitigate undesired phenomena arising when intentional jamming and false data injections, categorized as cyber-attacks, infer communication channels. Recent advances in sensing, communication and computing have open the door to the deployment of largescale networks of sensors and actuators that allow fine-grain monitoring and control of a multitude of physical processes and infrastructures. The appellation used by field experts for these paradigms is Cyber-Physical Systems (CPS) because the dynamics among computers, networking media/resources and physical systems interact in a way that multi-disciplinary technologies (embedded systems, computers, communications and controls) are required to accomplish prescribed missions. Moreover, they are expected to play a significant role in the design and development of future engineering applications such as smart grids, transportation systems, nuclear plants and smart factories.

This article considers the group consistency of second-order MAS with directly connected spanning tree communication topology. Because the MAS is divided into several groups, we proposed a group consistency control method based on intermittent control, and the range of parameters is given when the system achieves consensus. The protocol can realize periodic control and reduce the working hours of the controller in period. Furthermore, the group consistency of MAS is turn to the stability analysis of error, and a group consistency protocol of MAS with time-delays is designed. Finally, two examples are used for verify the theory.

Although quite a few results on resilient consensus of multi-agent systems with malicious agents and fixed topology have been reported in the literature, we lack any known results on such a problem for multi-agent systems with time-varying topologies. Herein, we study the resilient consensus problem of time-varying networked systems in the presence of misbehaving nodes. A novel concept of joint ( r, s) -robustness is firstly proposed to characterize the robustness of the time-varying topologies. It is further revealed that the resilient consensus of multi-agent systems under F-total malicious network can be reached by the Weighted Mean-Subsequence-Reduced algorithm if and only if the time-varying graph is jointly ( F+1, F+1) -robust. Numerical simulations are finally performed to verify the effectiveness of the analytical results.

The following topics are dealt with: security of data; distributed power generation; power engineering computing; power grids; power system security; computer network security; voltage control; risk management; power system measurement; critical infrastructures.

A distributed event-triggered controller is developed for formation control and leader tracking (FCLT) with robustness to adversarial Byzantine agents for a class of heterogeneous multi-agent systems (MASs). A reputation-based strategy is developed for each agent to detect Byzantine agent behaviors within their neighbor set and then selectively disregard Byzantine state information. Selectively ignoring Byzantine agents results in time-varying discontinuous changes to the network topology. Nonsmooth dynamics also result from the use of the event-triggered strategy enabling intermittent communication. Nonsmooth Lyapunov methods are used to prove stability and FCLT of the MAS consisting of the remaining cooperative agents.

In this paper, the privacy preserving average consensus problem of multi-agent systems with strongly connected and weight balanced graph is considered. In most existing consensus algorithms, the agents need to exchange their state information, which leads to the disclosure of their initial states. This might be undesirable because agents' initial states may contain some important and sensitive information. To solve the problem, we propose a novel distributed algorithm, which can guarantee average consensus and meanwhile preserve the agents' privacy. This algorithm assigns some additive perturbation signals on the communication edges and these perturbations signals will be added to original true states for information exchanging. This ensures that direct disclosure of initial states can be avoided. Then a rigid analysis of our algorithm's privacy preserving performance is provided. For any individual agent in the network, we present a necessary and sufficient condition under which its privacy is preserved. The effectiveness of our algorithm is demonstrated by a numerical simulation.

Computer networks and surging advancements of innovative information technology construct a critical infrastructure for network transactions of business entities. Information exchange and data access though such infrastructure is scrutinized by adversaries for vulnerabilities that lead to cyber-attacks. This paper presents an agent-based system modelling to conceptualize and extract explicit and latent structure of the complex enterprise systems as well as human interactions within the system to determine common vulnerabilities of the entity. The model captures emergent behavior resulting from interactions of multiple network agents including the number of workstations, regular, administrator and third-party users, external and internal attacks, defense mechanisms for the network setting, and many other parameters. A risk-based approach to modelling cybersecurity of a business entity is utilized to derive the rate of attacks. A neural network model will generalize the type of attack based on network traffic features allowing dynamic state changes. Rules of engagement to generate self-organizing behavior will be leveraged to appoint a defense mechanism suitable for the attack-state of the model. The effectiveness of the model will be depicted by time-state chart that shows the number of affected assets for the different types of attacks triggered by the entity risk and the time it takes to revert into normal state. The model will also associate a relevant cost per incident occurrence that derives the need for enhancement of security solutions.

The objective of this paper is to propose a model of a distributed intrusion detection system based on the multi-agent paradigm and the distributed file system (HDFS). Multi-agent systems (MAS) are very suitable to intrusion detection systems as they can address the issue of geographic data security in terms of autonomy, distribution and performance. The proposed system is based on a set of autonomous agents that cooperate and collaborate with each other to effectively detect intrusions and suspicious activities that may impact geographic information systems. Our system allows the detection of known and unknown computer attacks without any human intervention (Security Experts) unlike traditional intrusion detection systems that rely on knowledge bases as a mechanism to detect known attacks. The proposed model allows a real time detection of known and unknown attacks within large networks hosting geographic data.

Human behaviors are often prohibited, or permitted by social norms. Therefore, if autonomous agents interact with humans, they also need to reason about various legal rules, social and ethical social norms, so they would be trusted and accepted by humans. Inverse Reinforcement Learning (IRL) can be used for the autonomous agents to learn social norm-compliant behavior via expert demonstrations. However, norms are context-sensitive, i.e. different norms get activated in different contexts. For example, the privacy norm is activated for a domestic robot entering a bathroom where a person may be present, whereas it is not activated for the robot entering the kitchen. Representing various contexts in the state space of the robot, as well as getting expert demonstrations under all possible tasks and contexts is extremely challenging. Inspired by recent work on Modularized Normative MDP (MNMDP) and early work on context-sensitive RL, we propose a new IRL framework, Context-Sensitive Norm IRL (CNIRL). CNIRL treats states and contexts separately, and assumes that the expert determines the priority of every possible norm in the environment, where each norm is associated with a distinct reward function. The agent chooses the action to maximize its cumulative rewards. We present the CNIRL model and show that its computational complexity is scalable in the number of norms. We also show via two experimental scenarios that CNIRL can handle problems with changing context spaces.

The article is devoted to the analysis of the use of blockchain technology for self-organization of network communities. Network communities are characterized by the key role of trust in personal interactions, the need for repeated interactions, strong and weak ties within the network, social learning as the mechanism of self-organization. Therefore, in network communities reputation is the central component of social action, assessment of the situation, and formation of the expectations. The current proliferation of virtual network communities requires the development of appropriate technical infrastructure in the form of reputation systems - programs that provide calculation of network members reputation and organization of their cooperation and interaction. Traditional reputation systems have vulnerabilities in the field of information security and prevention of abusive behavior of agents. Overcoming these restrictions is possible through integration of reputation systems and blockchain technology that allows to increase transparency of reputation assessment system and prevent attempts of manipulation the system and social engineering. At the same time, the most promising is the use of blockchain-oracles to ensure communication between the algorithms of blockchain-based reputation system and the external information environment. The popularization of blockchain technology and its implementation in various spheres of social management, production control, economic exchange actualizes the problems of using digital technologies in political processes and their impact on the formation of digital authoritarianism, digital democracy and digital anarchism. The paper emphasizes that blockchain technology and reputation systems can equally benefit both the resources of government control and tools of democratization and public accountability to civil society or even practices of avoiding government. Therefore, it is important to take into account the problems of political institutionalization, path dependence and the creation of differentiated incentives as well as the technological aspects.

Cyber-Physical Systems (CPS) are monitored and controlled by Supervisory Control and Data Acquisition (SCADA) systems that use advanced computing, sensors, control systems, and communication networks. At first, CPS and SCADA systems were protected and secured by isolation. However, with recent industrial technology advances, the increased connectivity of CPSs and SCADA systems to enterprise networks has uncovered them to new cybersecurity threats and made them a primary target for cyber-attacks with the potential of causing catastrophic economic, social, and environmental damage. Recent research focuses on new methodologies for risk modeling and assessment using game theory and reinforcement learning. This paperwork proposes to frame CPS security on two different levels, strategic and battlefield, by meeting ideas from game theory and Multi-Agent Reinforcement Learning (MARL). The strategic level is modeled as imperfect information, extensive form game. Here, the human administrator and the malware author decide on the strategies of defense and attack, respectively. At the battlefield level, strategies are implemented by machine learning agents that derive optimal policies for run-time decisions. The outcomes of these policies manifest as the utility at a higher level, where we aim to reach a Nash Equilibrium (NE) in favor of the defender. We simulate the scenario of a virus spreading in the context of a CPS network. We present experiments using the MiniCPS simulator and the OpenAI Gym toolkit and discuss the results.

We study the problem of designing a dynamic mechanism for security management in an interconnected multi-agent system with N strategic agents and one coordinator. The system is modeled as a network of N vertices. Each agent resides in one of the vertices of the network and has a privately known security state that describes its safety level at each time. The evolution of an agent's security state depends on its own state, the states of its neighbors in the network and on actions taken by a network coordinator. Each agent's utility at time instant t depends on its own state, the states of its neighbors in the network and on actions taken by a network coordinator. The objective of the network coordinator is to take security actions in order to maximize the long-term expected social surplus. Since agents are strategic and their security states are private information, the coordinator needs to incentivize agents to reveal their information. This results in a dynamic mechanism design problem for the coordinator. We leverage the inter-temporal correlations between the agents' security states to identify sufficient conditions under which an incentive compatible expected social surplus maximizing mechanism can be constructed. We then identify two special cases of our formulation and describe how the desired mechanism is constructed in these cases.

A trust model IMPACT: Intention, Measurability, Predictability, Agility, Communication, and Transparency has been conceptualized to build human trust in autonomous agents. The six critical characteristics must be exhibited by the agents in order to gain and maintain the trust from their human partners towards an effective and collaborative team in achieving common goals. The IMPACT model guided a design of an intelligent adaptive decision aid for dynamic target engagement processes in a military context. Positive feedback from subject matter experts participated in a large scale joint exercise controlling multiple unmanned vehicles indicated the effectiveness of the decision aid. It also demonstrated the utility of the IMPACT model as design principles for building up a trusted human-agent teaming.

How does information regarding an adversary's intentions affect optimal system design? This paper addresses this question in the context of graphical coordination games where an adversary can indirectly influence the behavior of agents by modifying their payoffs. We study a situation in which a system operator must select a graph topology in anticipation of the action of an unknown adversary. The designer can limit her worst-case losses by playing a security strategy, effectively planning for an adversary which intends maximum harm. However, fine-grained information regarding the adversary's intention may help the system operator to fine-tune the defenses and obtain better system performance. In a simple model of adversarial behavior, this paper asks how much a system operator can gain by fine-tuning a defense for known adversarial intent. We find that if the adversary is weak, a security strategy is approximately optimal for any adversary type; however, for moderately-strong adversaries, security strategies are far from optimal.

Consensus-based distributed microgrid energy management system is one of the most used distributed control strategies in the microgrid area. To improve its cybersecurity, the system needs to evaluate the trustworthiness of the participating agents in addition to the conventional cryptography efforts. This paper proposes a novel augmented reputation metric to evaluate the agents' trustworthiness in a distributed fashion. The proposed metric adopts a novel augmentation method to substantially improve the trust evaluation and attack detection performance under three typical difficult-to-detect attack patterns. The proposed metric is implemented and validated on a real-time HIL microgrid testbed.

With the recent advances in computing, artificial intelligence (AI) is quickly becoming a key component in the future of advanced applications. In one application in particular, AI has played a major role - that of revolutionizing traditional healthcare assistance. Using embodied interactive agents, or interactive robots, in healthcare scenarios has emerged as an innovative way to interact with patients. As an essential factor for interpersonal interaction, trust plays a crucial role in establishing and maintaining a patient-agent relationship. In this paper, we discuss a study related to healthcare in which we examine aspects of trust between humans and interactive robots during a therapy intervention in which the agent provides corrective feedback. A total of twenty participants were randomly assigned to receive corrective feedback from either a robotic agent or a human agent. Survey results indicate trust in a therapy intervention coupled with a robotic agent is comparable to that of trust in an intervention coupled with a human agent. Results also show a trend that the agent condition has a medium-sized effect on trust. In addition, we found that participants in the robot therapist condition are 3.5 times likely to have trust involved in their decision than the participants in the human therapist condition. These results indicate that the deployment of interactive robot agents in healthcare scenarios has the potential to maintain quality of health for future generations.

Network security is an important issue in today's world with existence of network systems that communicate data and information about all aspects of our life, work and business. Network security is an important issue with connected networks and data communication between organisations of that specialized in different areas. Network security engineers spend a considerable amount of time to investigate network for security breaches and to enhance the security of their networks and data communications on their networks. They use Attack Graphs (AGs) which are graphical representation of networks to assist them in analysing large networks. With increase size of networks and their complexity, the use of attack graphs alone does not provide the necessary risk analysis and assessment facilities. There is a need for automated intelligent systems such as multiagent systems to assist in analysing, assessing and testing networks. Network systems changes with the increase in the size of organisation and connectivity of network of organisations based on the business needs or organisational or governmental rules and regulations. In this paper a multi-agent system is developed assist in analysing interconnected network to identify security risks. The multi-agent system is capable of security network analysis to identify paths using an attack graph of the network under consideration to protect network systems, as the networks grow and change, against possible attacks. The multiagent system uses a model developed by Mohammadian [3] for converting AGs to Fuzzy Cognitive Maps (FCMs) to identify attack paths from attack graphs and perform security risk analysis. In this paper a novel decision-making approach using FCMs is employed.

Modern computing networks that enable distributed computing are comprised of a wide range of heterogeneous devices with different levels of resources, which are interconnected by different networking technologies and communication protocols. This integration, together with the state of the art technologies, has brought into play new uncertainties, associated with physical world and the cyber space. In heterogeneous networked control systems environments, awareness and resilience are two important properties that these systems should bear and comply with. In this work the problem of resilience enhancement in heterogeneous networked control systems is addressed based on a distributed middleware, which is propped up on a hierarchical multi-agent framework, where each of the constituent agents is devoted to a specific task. The proposed architecture takes into account physical and cyber vulnerabilities and ensures state and context awareness, and a minimum level of acceptable operational performance, in response to physical and cyber disturbances. Experiments on a IPv6-based test-bed proved the relevance and benefits offered by the proposed architecture.

Advances on the integration of wireless sensor and actuator networks, as a whole, have contribute to the greater reconfigurability of systems and lower installation costs with application to supervision of networked control systems. This integration, however, increases some vulnerabilities associated with the physical world and also with the cyber and security world. This trend makes the wireless nodes one of the most vulnerable component of these kind of systems, which can have a major impact on the overall performance of the networked control system. This paper presents an architecture relying on a hierarchical multi-agent system for resilience enhancement, with focus on wireless sensor and actuator networks. The proposed framework was evaluated on an IPv6 test-bed comprising several distributed devices, where performance and communication links health are analyzed. The relevance of the proposed approach is demonstrated by results collected from the test-bed.