Biblio

While the existence of many security elements in software can be measured (e.g., vulnerabilities, security controls, or privacy controls), it is challenging to measure their relative security impact. In the physical world we can often measure the impact of individual elements to a system. However, in cyber security we often lack ground truth (i.e., the ability to directly measure significance). In this work we propose to solve this by leveraging human expert opinion to provide ground truth. Experts are iteratively asked to compare pairs of security elements to determine their relative significance. On the back end our knowledge encoding tool performs a form of binary insertion sort on a set of security elements using each expert as an oracle for the element comparisons. The tool not only sorts the elements (note that equality may be permitted), but it also records the strength or degree of each relationship. The output is a directed acyclic ‘constraint’ graph that provides a total ordering among the sets of equivalent elements. Multiple constraint graphs are then unified together to form a single graph that is used to generate a scoring or prioritization system.For our empirical study, we apply this domain-agnostic measurement approach to generate scoring/prioritization systems in the areas of vulnerability scoring, privacy control prioritization, and cyber security control evaluation.

Cyber-security training has evolved into an imperative need, aiming to provide cyber-security professionals with the knowledge and skills required to confront cyber-attacks that are increasing in number and sophistication. Training activities are typically associated with evaluation means, aimed to assess the extent to which the trainee has acquired the knowledge and skills whose development is targeted by the training programme, while cyber-security awareness and skill level evaluation means may be used to support additional security-related aspects of organizations. In this paper, we review trainee performance assessment metrics in cyber-security training, aiming to assist designers of cyber-security training activities to identify the most prominent trainee performance assessment means for their training programmes, while additional research directions involving cyber-security training evaluation metrics are also identified.