Multi-model Testbed for the Simulation-based Evaluation of Resilience

PI(s), Co-PI(s), Researchers:

• Peter Volgyesi (PI)
• Himanshu Neema (Co-PI)

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released November 2012.

• Security Metrics Driven Evaluation, Design, Development, and Deployment
• Resilient Architectures

The goal of the Multi-model Testbed is to provide a collaborative design tool for evaluating various cyber attack/defense strategies and their effects on the physical infrastructure. The web-based, cloud-hosted environment integrates state-of-the-art simulation engines for the different CPS domains and presents interesting research challenges as ready to use scenarios. Input data, model parameters, and simulation results are archived, versioned with a strong emphasis on repeatability and provenance. 

PUBLICATIONS

Akos Ledeczi, Miklos Maroti, Peter Volgyesi, Brian Broll, Hamid Zare, Bernard Yett, Timothy Darrah, Mary Metelko, Nicole Hutchins, Michael Smith, Gautam Biswas and Xenofon Koutsoukos. "Teaching Cybersecurity with Networked Robots". Special Interest Group on Computer Science Education (SIGCSE) Technical Symposium 2019, February 27th - March 2nd, Minneapolis, Minnesota, USA.

Bradley Potteiger, Zhenkai Zhang and Xenofon Koutsoukos.  "Integrated Data Space Randomization and Control Reconfiguration for Securing Cyber-Physical Systems". 6th Annual Hot Topics in the Science of Security, HotSoS 2019, April 2-3, Nashville, TN, USA.

Carlos Barreto and Xenofon Koutsoukos "Attacks on Electricity Markets", 6th Annual Hot Topics in the Science of Security, HotSoS 2019, April 2-3, Nashville, TN, USA (work-in-progress presentation).

Bradley Potteiger, Hamzah Abdel-Aziz, Himanshu Neema, Xenofon Koutsoukos, "Simulation Based Evaluation of Security and Resilience in Railway Infrastructure", 6th Annual Hot Topics in the Science of Security, HotSoS 2019, April 2-3, Nashville, TN, USA (poster).

Xingyu Zhou, Yi Li, Carlos Barreto, Peter Volgyesi and Xenofon Koutsoukos "Load Forecasting With Adversarial Attacks In Power Systems Using DeepForge", 6th Annual Hot Topics in the Science of Security, HotSoS 2019, April 2-3, Nashville, TN, USA (poster).

Himanshu Neema and Harsh Vardhan "Design and Simulation Platform for Evaluation of Grid Distribution System and Transactive Energy", 6th Annual Hot Topics in the Science of Security, HotSoS 2019, April 2-3, Nashville, TN, USA (poster).

KEY HIGHLIGHTS

During the reporting period, we continued the development of two CPS domains for science of security experiments. In the future, these experiments will be refined and integrated in the web-based collaborative design environment and will be provided as security research scenarios.

Transportation domain

In the transportation domain our simulation and testbed efforts were focusing on two major topics: (1) applying and evaluating data-space randomization techniques for securing autonomous vehicle  platforms against non-control data attacks and (2) developing a hardware-in-the-loop (HIL) based simulation platform for investigating the effects and potential defense strategies against cyber attacks in railway transportation networks. Note, that both topics required theoretical and research efforts beyond the scope of this sub-project (Multi-model Testbed for the Simulation-based Evaluation of Resilience), which aims at developing tools and infrastructure for supporting such research. The preliminary results were presented at HotSos 2019.

Autonomous Vehicles

This simulation environment includes BeagleBone Black-based ECUs (2), an NVIDIA Jetson X2-based controller, a software-based physics engine (TORCS) and a web-based dashboard, using Grafana and InfluxDB.

Railway Networks

The following figures show the system architecture and hardware setup for running HIL-based simulations in the railway CPS domain.

Powergrid domain  We continued our work on supporting the smart grid CPS domain, based on the GridLAB-D simulator and machine-learning frameworks. Work with the low-level power distribution system simulator is  supported by a WebGME-based Design Studio, which provides a web-based graphical interface with rich collaboration and version management facilities for developing simulation scenarios. The generated datasets are used to develop machine learning-based methods for predicting future power demand along with adversarial attacks against smart meter readings to influence the predicted load. For developing the prediction models, attack strategies and potential detection and mitigation algorithms we are using the TensorFlow and Keras frameworks supported by another in-house developed WebGME-based Design Studio, called DeepForge. The current attack strategy uses a gradient-based approach for attacking a (sub)set of most sensitive/influential sensors, whereas the initial defense strategy utilizes random dropouts at the input layer. The preliminary results and the supporting infrastructure were presented at HotSoS 2019.

The following figure shoews the load prediction performance using RNN with three Long-Short-Term Memory (LSTM) layers(150 units) and two fully connected layers (500 nodes)
 

The mean-squared-error (MSE) is shown with different (levels of) L0-FGSM attacks. Deviation is the relative amount of perturbation on the sensor readings (attack).

Testbed Infrastructure 

We continued our work on the WebGME-based  web-based front-end and Jupyter notebooks-based analysis environment. The current infrastructure of the testbed is available at http://lablet.isis.vanderbilt.edu. Note, that the current version does not provide developed scenario models. 

COMMUNITY ENGAGEMENTS

None, during this reporting period.

EDUCATIONAL ADVANCES:

None, during this reporting period.