Multi-model Testbed for the Simulation-based Evaluation of Resilience (July '19)

PI(s), Co-PI(s), Researchers:

• Peter Volgyesi (PI)
• Himanshu Neema (Co-PI)

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released in November 2012.

• Security Metrics Driven Evaluation, Design, Development, and Deployment
• Resilient Architectures

The goal of the Multi-model Testbed is to provide a collaborative design tool for evaluating various cyber attack/defense strategies and their effects on the physical infrastructure. The web-based, cloud-hosted environment integrates state-of-the-art simulation engines for the different CPS domains and presents interesting research challenges as ready to use scenarios. Input data, model parameters, and simulation results are archived, versioned with a strong emphasis on repeatability and provenance.

PUBLICATIONS

Xingyu Zhou, Yi Li, Carlos Barreto, Jiani Li, Peter Volgyesi, Himanshu Neema, Xenofon Koutsoukos, "Resilient Forecasting of Grid Loads under Stealthy Adversarial Attacks". Resilience Week 2019 Symposium, San Antonio, TX, USA, November 4-7, 2019. (submitted)

Bradley Potteiger, Himanshu Neema, Xenofon Koutsoukos, Chee Yee Tang, Keith Stouffer, "Simulation Based Rapid Evaluation Platform for Security and Resilience in Railway Infrastructure". Resilience Week 2019 Symposium, San Antonio, TX, USA, November 4-7, 2019. (submitted)

KEY HIGHLIGHTS

During the reporting period, our testbed efforts were primarily focusing on developing a fully integrated workflow in DeepForge targeting the smart grid CPS domain. This work had two major goals. (1) A complete set of prediction, attack and detection models have been developed for load forecasting applications. (2) Several building blocks - most notably for gradient-based deep neural network attacks - of these models are generalized to form the basis of a future library of reusable components to create SoS experiments involving learning enabled components.

The individual workflows (pipelines) developed for the GridLAB-D-based simulation results are shown in the next figure.

Each pipeline in the previous figure is a DAG (directed acyclic graph) of operations. These operations are implemented in Python, but the DeepForge model captures the input/output interfaces and run-time parameters of these steps. It also provides an integrated web-based code editor for developing the executable script behind each operation. While developing these building blocks we considered future re-use and generalization. The pipeline model and its basic operations are shown in the next figure (IGSM_stealth, ScoreModel, GetDeviation are generic blocks):

The DeepForge-based testbed provides automatic versioning for the models and the input/output artifacts of each run of the workflow model(s). The status of previously and currently executed workflows can be tracked real-time through the web interface. The following figure shows the execution status of the previous pipeline.

Apart from the high-level status, the textual (console) output and/or visual plots of each operation is being captured and available through the interface. The following figure shows the progress of the loss function through the training process.

The current testbed, the developed examples, operations and DNN network architectures are hosted on our cloud infrastructure and available online at http://lablet.isis.vanderbilt.edu (password protected, please contact the PIs if access is

FUTURE PLANS

A key missing element of the current testbed is the integration of third-party simulators of various CPS domains with the DeepForge-based workflows. Currently, these simulators are run before/outside of the web-based experiments. In the past, we used WebGME, the same foundational technology which drives DeepForge, for developing web-based front-ends for SUMO and GridLAB-D simulations. Based on our current results and experiences with the DeepForge workflow model, we are convinced we can rely on the pipeline mechanism for integrating domain-specific models and simulators with DeepForge. The following figure shows the high-level architecture of this planned effort. We will provide a more detailed description at the next quarterly Lablet meeting.

COMMUNITY ENGAGEMENTS, TECHNOLOGY TRANSFER

In the last six month, we developed a novel domain-specific framework for analyzing the security and resilience of railway infrastructures. The framework has an integrated cloud backend that supports simulating railway transportation and cyber communication networks in an integrated manner. The framework provides a web-based graphical modeling environment to rapidly develop attack scenarios against railway networks. The simulation infrastructure supports collecting key operational metrics directly relevant in the railway domain and a dashboard for real-time visualization of experimental results. Moreover, we have integrated a hardware-in-the-loop (HIL) testbed for evaluating specific railway scenarios against a variety of realistic cyber-attacks. The HIL testbed provides hardware configurations consistent with the railway systems deployment environment. We also developed a case study using the Washington DC Metro railway network and demonstrated the framework’s capabilities using it. We have submitted a research paper describing the framework architecture as well as the experimental results. As part of our ongoing work with National Institute of Standards and Technology (NIST), the entire software has been transitioned in their laboratory and is being actively developed and supported by us at Vanderbilt. For evaluating the performance impact of cybersecurity mechanisms on industrial control systems (ICS), NIST has created a unique testbed that emulates real-world scenarios using software simulators and commercial control hardware in the laboratory environment. This work was transitioned to the Intelligent Systems Division of the Engineering Laboratory at the NIST Gaithersburg campus. In particular, the Network Control Systems Group at NIST has a HIL testbed to emulate railroad crossings using a commercial Programmable Logic Controller (PLC), an industrial network switch, and two embedded sensors. The PLC communicates with the sensors via a Controller Area Network (CAN) interface and with our railway simulation environment via TCP/IP sockets. In coordination with several companies over the last several years, NIST has developed detailed specifications for a cybersecurity framework. This provides guidance for the public and private sectors on how to secure industrial control systems while addressing their unique performance, reliability, and safety requirements. The transitioned work is aimed at helping NIST in further developing its cybersecurity framework specifications and recommendations.

EDUCATIONAL ADVANCES:

None, during this reporting period.