VERIFICATION OF CYBER EMULATION EXPERIMENTS

involve large numbers of parallel emulations meant to oversubscribe resources at some point. We present an approach to use telemetry metrics for emulation verification, and we demonstrate this approach on two cyber scenarios. Descriptions of the experimental configurations are provided along with a detailed discussion of statistical tests used to compare telemetry metrics. Results demonstrate the potential for a structured experimental framework, combined with statistical analysis of telemetry metrics, to support emulation verification.We conclude with comments on generalizability and potential future work.

Jamie Thorpe is a cybersecurity researcher at Sandia National Laboratories in Albuquerque, New Mexico, where she works to develop tools needed to help build and analyze models of cyber-physical systems, such as power systems. Her research interests include cyber resilience metrics, system model development, data analysis for emulated environments, and emulation verification.

Laura Swiler is a computational scientist at Sandia National Laboratories whose research focuses on quantifying the uncertainty associated with predictions from models. Her research interests include experimental design, sampling algorithms, Bayesian inference, and surrogate models. Laura has worked on many application areas, including nuclear waste repository assessment, circuit model calibration, additive manufacturing, and cyber emulation.

Thomas Tarman is a distinguished member of the technical staff at Sandia National Laboratories in Albuquerque, New Mexico, where he leads research in virtualization and rigorous cyber experimentation methodologies, with application to high-consequence cyber systems. His research interests are in network modeling and simulation, hybrid simulation-emulation-physical testbeds for cyber-security research, and network security protocols.