Multi-model Testbed for the Simulation-based Evaluation of Resilience (October '20)

PI(s), Co-PI(s), Researchers:

• Peter Volgyesi (PI)
• Himanshu Neema (Co-PI)

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released in November 2012.

• Security Metrics Driven Evaluation, Design, Development, and Deployment
• Resilient Architectures

The goal of the Multi-model Testbed is to provide a collaborative design tool for evaluating various cyber-attack / defense strategies and their effects on the physical infrastructure. The web-based, cloud-hosted environment integrates state-of-the-art simulation engines for the different CPS domains and presents interesting research challenges as ready to use scenarios. Input data, model parameters, and simulation results are archived, versioned with a strong emphasis on repeatability and provenance.

PUBLICATIONS

[1] Himanshu Neema, Xenofon Koutsoukos, Bradley Potteiger, CheeYee Tang, and Keith Stouffer. 2020. Simulation testbed for railway infrastructure security and resilience evaluation. In Proceedings of the 7th Symposium on Hot Topics in the Science of Security (HotSoS '20). Association for Computing Machinery, New York, NY, USA, Article 1, 1–8. DOI:https://doi.org/10.1145/3384217.3385623 (Best Paper Award)

[2] Ali I Ozdagli, Carlos Barreto, and Xenofon Koutsoukos. 2020. @PAD: adversarial training of power systems against denial-of-service attacks. In Proceedings of the 7th Symposium on Hot Topics in the Science of Security (HotSoS '20). Association for Computing Machinery, New York, NY, USA, Article 2, 1–9. DOI:https://doi.org/10.1145/3384217.3385616

[3] Jiani Li, Waseem Abbas, Xenofon Koutsoukos, and Mudassir Shabbir, Resilient Multi-Robot Target Pursuit. 7th Symposium on Hot Topics in the Science of Security (HotSoS '20). Virtual symposium. September 22-24, 2020. (poster)

[4] Jiani Li, Waseem Abbas, Xenofon Koutsoukos, Byzantine Resilient Distributed Multi-Task Learning, Thirty-fourth Annual Conference on Neural Information Processing Systems (NeurIPS 2020). Virtual conference. Dec 6-12, 2020. (accepted)

KEY HIGHLIGHTS

Threat Modeling and Risk Analysis in Industrial Control Systems

In this effort, we are working on developing a modeling and analysis framework for threats and cybersecurity risks in Industrial Control Systems (ICS). Identification of system vulnerabilities and implementation of appropriate risk mitigation strategies are crucial for ensuring the cybersecurity of Industrial Control Systems (ICS). These system vulnerabilities must be evaluated depending on their exploitability, impact, mitigation status, and target platform and environments. Therefore, in order to assess system vulnerabilities and risk mitigation strategies quantitatively, we are focusing on threat modeling and risk analysis methods for the cybersecurity of Railway Transportation Systems (RTS), which are real-world ICS and have become increasing vulnerable to cyber-attacks due to growing reliance on networked physical and computation components. The first task in this work is designing the required modeling language that will enable modeling of: (a) the physical and network infrastructure of the system, (b) vulnerabilities in system components, (c) component attack trees that describe different ways in which specific component attacks could be carried out, and (d) attack graphs that describe how attacks can propagate across multiple system components. Secondly, we also need to develop a set of analysis tools for interpreting the models and assessing system risk against different attack vectors. In the last quarter, we have developed the modeling language for basic architecture modeling as well as vulnerability propagation. However, modeling risk propagation across multiple system components is still an ongoing work. As the task underwent, we have also now planned to develop visualization tools for the generated attack trees as well as designing several use-cases that demonstrate the effectiveness of this model-based and intuitive risk analysis approach for CPS.

Domain-specific language (DSL) for scenario-generation

In our simulation-based research experiments such as for cybersecurity evaluation of CPS, as well as for designing machine learning based intelligent and decision support agents, we need a way to generate large amounts of "relevant" data through simulations. In order to do that, we need to come up with a specification for not only the different scenarios but also for parametric variations of those scenarios to model different external conditions under which the system must be evaluated. We call this method as scenario generation. For example, consider an autonomous driving vehicle system that has been trained using machine learning models. In these systems, various situations arise that are highly interesting from the point of view of where things can go wrong. Secondly, most of these situations can often be figured out through algorithms that aim to maximize learning of the autonomous agents while minimizing the cost of training through a large number of scenarios. As a first step, we need to design a domain-specific language to be able to model and specify the scenarios and scenario variations (i.e., a modeling language for scenario generation). In addition, we need the methods and tools to be able to convert these scenario specifications into runnable simulation experiments. We are currently evaluating a simple textX tool for designing the scenario generation language..

Resilient Distributed Multi-Task Learning

We advanced our research for distributed learning in multi-agent networks. Specifically, in [3] Jiani et al. analyzed and presented an approach for distributed multi-task learning in the presence of Byzantine agents using a novel online weight adjustment strategy. The evaluation of the research relied on multiple datasets with traditional machine learning models (linear models for target localization and human activity recognition)  and deep learning models (for digit classification).

Physics-guided Machine Learning of Surrogate Models

We also continued our work on Physics-guided deep neural network training (PhyDNNs), targeting the structural health monitoring and assessment domain. In the experimentation efforts, we currently use the OpenSees simulation tool to detect, localize, and quantify the damage on the structure. The obtained results - from long-running FEM simulation runs - are used for training and evaluating machine learning models to shortcut future analysis tasks and/or to provide a differentiable model from input parameters to simulation results. 

Deep Learning Testbed Infrastructure

Deep Learning is one of the fundamental methods we rely on in our Lablet research [2][3][4]. The testbed supports these efforts via Deepforge, our collaborative model-based environment for designing and evaluating complete machine learning workflows.  Constructing such machine learning pipelines can be an effective abstraction for promoting reusable components and executing long-running tasks. However, this approach can be inconvenient for simple, exploratory tasks where quick, immediate feedback is preferred. Interactivity and immediate feedback have already proven to be powerful mechanisms for promoting accessibility, as evinced by their success in both novice programming environments and common data science environments (a good example is the Jupyter Notebook environment). The ability to reconstruct an entire history in the form of a pipeline from an artifact makes a compelling case for the interactive, implicit construction of pipelines.

To support these kinds of exploratory development workflows we added a new interactive editing capability to Deepforge.
Interactive editing in DeepForge is supported through the introduction of ``interactive compute sessions,'' along with visual editors. An interactive compute session is a connection to a compute resource that can be used for immediate execution of arbitrary commands and is a simple extension of the existing compute infrastructure. Using compute adapters, a custom job is created which establishes a websocket connection to the browser (proxied through the DeepForge server). A bidirectional communication channel supports a relatively simple API which includes importing artifacts or files into the session and spawning processes. Similarly to Jupyter Notebook, the underlying job only executes a single command at a time. Using interactive compute capabilities, visual editors can inspect and create artifacts. 

Interactive Data Exploration in Deepforge

However, arbitrary interactive editing and code execution can make reproducing workflows difficult. 
Furthermore, we would like to enable the creation of the provenance of an artifact as an executable pipeline
To this end, we introduce the concept of Implicit Operations. The workflow steps are created and edited by visual editors during the interactive sessions. While capturing the versioned data links and parameters from the interactive exploration phase, these steps can be used in future pipeline models.

Implicit Operations - Preserving and Embedding Interactive Exploration Results in Pipelines

Virtual technology workshop with Fujitsu System Integration Laboratories, Ltd., August 27, 2020.

Discussion topics:

• Fujitsu Cyber Range Builder
• Fujitsu Cyber Threat Intelligence Tools
• MITRE CALDERA* for Cyber exercise coupled to Cyber Range Builder
• Vanderbilt WebGME and Deepforge integration with Fujitsu security testbeds

Collaboration with NIST on threat modeling and risk analysis in ICS

Discussion topics:

• Threat modeling in Railway ICS
• Risk Analysis
• Quantitative Risk Evaluation
• Integration with Simulation-Based Evaluation

EDUCATIONAL ADVANCES and OUTREACH

Dr. Himanshu Neema is currently advising three undergraduate students for their Fall internships at our institute. These internship projects are described below. Please note that these students are working with our technologies, but the internships are not funded by this project.

Local Virtual Power Plant with Transactive Energy

This project aims to evaluate whether transactive energy can provide economic benefit for developing virtual power plants (VPPs) for cities. In VPPs, communities transition from using more electricity for locally generated energy from renewable energy sources such as rooftop solar panels. However, this is highly cost prohibitive and so this project is investigating whether TE can reduce these costs. In addition, the project also aims to evaluate the impact of TE on better management of supply and demand in the DER integrated distribution grid.

Threat Modeling and Risk Analysis for ICS

This project aims to develop modeling language and analysis tools for CPS. The language is being developed using Vanderbilt's WebGME modeling environment. Using the modeling language one can create system architecture for specific ICS such as railway infrastructure. The system architecture comprises of various components within the system and different types of network connectivity among these components. Further, vulnerability scores could be assigned to various components with regard to different categories of Microsoft's STRIDE threat modeling method. The analysis tools as part of the language could be used to calculate vulnerability scores at the component level. In addition, the visualization tools will help show the generated component attack trees and system attack graphs that directly help with risk analysis of ICS.

Evaluation of Vector Control and Social Policies on Pathogen Spread within Communities

This recently started project aims to utilize agent-based simulations for modeling arthropod behavior and human activities as well as social policies for vector control and for changing human behavior in order to evaluate how these affect the spread of pathogens in humans through mosquito bites. We plan to use integrated simulations for these evaluations.