Biblio

The growing complexity of server architectures, which incorporate several components with state, has necessitated rigorous assessment of the security risk both during design and operation. In this paper, we propose a novel technique to model the security risk of servers by mapping their architectures to graphs. This allows us to leverage tools from computational graph theory, which we combine with probability theory for deriving quantitative metrics for risk assessment. Probability of attack is derived for server components, with prior probabilities assigned based on knowledge of existing vulnerabilities and countermeasures. The resulting analysis is further used to compute measures of impact and exploitability of attack. The proposed methods are demonstrated on two open-source server designs with different architectures.

Many popular metrics used for the quantification of the quality or complexity of a codebase (e.g. cyclomatic complexity) were developed in the 1970s or 1980s when source code sizes were significantly smaller than they are today, and before a number of modern programming language features were introduced in different languages. Thus, the many thresholds that were suggested by researchers for deciding whether a given function is lacking in a given quality dimension need to be updated. In the pursuit of this goal, we study a number of open-source high-performance codes, each of which has been in development for more than 15 years—a characteristic which we take to imply good design to score them in terms of their source codes' quality and to relax the above-mentioned thresholds. First, we employ the LLVM/Clang compiler infrastructure and introduce a Clang AST tool to gather AST-based metrics, as well as an LLVM IR pass for those based on a source code's static call graph. Second, we perform statistical analysis to identify the reference thresholds of 22 code quality and callgraph-related metrics at a fine grained level.

Probabilistic security metrics estimate the vulnerability of a network in terms of the likelihood of an attacker reaching the goal states (of a network) by exploiting the attack graph paths. The probability computation depends upon several assumptions regarding the possible attack scenarios. In this paper, we extend the existing security metric to model networks with intrusion detection systems and their associated uncertainties and time latencies. We consider learning capabilities of attackers as well as detection systems. Estimation of risk is obtained by using the attack paths that are undetectable owing to the latency of the detection system. Thus, we define the overall vulnerability (of a network) as a function of the time window available to an attacker for repeated exploring (via learning) and exploitation of a network, before the attack is mitigated by the detection system. Finally, we consider the realistic scenario where an attacker explores and abandons various partial paths in the attack graph before the actual exploitation. A dynamic programming formulation of the vulnerability computation methodology is proposed for this scenario. The nature of these metrics are explained using a case study showing the vulnerability spectrum from the case of zero detection latency to a no detection scenario.

During this period of high-speed internet, there are a number of serious challenges for software security protection of software design, especially throughout the life cycle of the process of software design, in which there are various risks involving information interaction. Significant information leakage can result from a lack of technical support and software security protection. One major problem with regard to creating software that includes security is the way that secure software is defined and the methods that are used for the measurement of security. The point of this research work is on the software engineers' perspective regarding security in the stage of software design. The tools for the measurement of the metrics are employed for the evaluation of the software's security. In this case study, a metric category of design are used, which are assumed to provide quantitative data about the software's security.

The increasingly intensive use of software systems in diverse sectors, especially in business, government, healthcare, and critical infrastructures, makes it essential to deliver code that is secure. In this work, we present two sets of experiments aiming at helping developers to improve software security from the early development stages. The first experiment is focused on using software metrics to build prediction models to distinguish vulnerable from non-vulnerable code. The second experiment studies the hypothesis of developing a consensus-based decision-making approach on top of several machine learning-based prediction models, trained using software metrics data to categorize code units with respect to their security. Such categories suggest a priority (ranking) of software code units based on the potential existence of security vulnerabilities. Results show that software metrics do not constitute sufficient evidence of security issues and cannot effectively be used to build a prediction model to distinguish vulnerable from non-vulnerable code. However, with a consensus-based decision-making approach, it is possible to classify code units from a security perspective, which allows developers to decide (considering the criticality of the system under development and the available resources) which parts of the software should be the focal point for the detection and removal of security vulnerabilities.

Implementations of Cyber-Physical Systems (CPS), like the Internet of Things, Smart Factories or Smart Grid gain more and more impact in their fields of application, as they extend the functionality and quality of the offered services significantly. However, the coupling of safety-critical embedded systems and services of the cyber-space domain introduce many new challenges for system engineers. Especially, the goal to achieve a high level of security throughout CPS presents a major challenge. However, it is necessary to develop and deploy secure CPS, as vulnerabilities and threats may lead to a non- or maliciously modified functionality of the CPS. This could ultimately cause harm to life of involved actors, or at least sensitive information can be leaked or lost. Therefore, it is essential that system engineers are aware of the level of security of the deployed CPS. For this purpose, security metrics and security evaluation frameworks can be utilized, as they are able to quantitatively express security, based on different measurements and rules. However, existing security scoring solutions may not be able to generate accurate security scores for CPS, as they insufficiently consider the typical CPS characteristics, like the communication of heterogeneous systems of physical- and cyber-space domain in an unpredictable manner. Therefore, we propose a security analysis framework, called Security Qualification Matrix (SQM). The SQM is capable to analyses multiple attacks on a System-of-Systems level simultaneously. With this approach, dependencies, potential side effects and the impact of mitigation concepts can quickly be identified and evaluated.

Development of quality object-oriented software contains security as an integral aspect of that process. During that process, a ceaseless burden on the developers was posed in order to maximize the development and at the same time to reduce the expense and time invested in security. In this paper, the authors analyzed metrics for object-oriented software in order to evaluate and identify the relation between metric value and security of the software. Identification of these relations was achieved by study of software vulnerabilities with code level metrics. By using OWASP classification of vulnerabilities and experimental results, we proved that there was relation between metric values and possible security issues in software. For experimental code analysis, we have developed special software called SOFTMET.

The coupling of safety-relevant embedded- and cyber-space components to build Cyber-Physical Systems (CPS) extends the functionality and quality in many business domains, while also creating new ones. Prime examples like Internet of Things and Industry 4.0 enable new technologies and extend the service capabilities of physical entities by building a universe of connected devices. In addition to higher complexity, the coupling of these heterogeneous systems results in many new challenges, which should be addressed by engineers and administrators. Here, security represents a major challenge, which may be well addressed in cyber-space engineering, but less in embedded system or CPS design. Although model-based engineering provides significant benefits for system architects, like reducing complexity and automated analysis, as well as being considered as standard methodology in embedded systems design, the aspect of security may not have had a major role in traditional engineering concepts. Especially the characteristics of CPS, as well as the coupling of safety-relevant (physical) components with high-scalable entities of the cyber-space domain have an enormous impact on the overall level of security, based on the introduced side effects and uncertainties. Therefore, we aim to define a model-based security-engineering framework, which is tailored to the needs of CPS engineers. Hereby, we focus on the actual modeling process, the evaluation of security, as well as quantitatively expressing security of a deployed CPS. Overall and in contrast to other approaches, we shift the engineering concepts on a semantic level, which allows to address the proposed challenges in CPS in the most efficient way.

Measuring software complexity is key in managing the software lifecycle and in controlling its maintenance. While there are well-established and comprehensive metrics to measure the complexity of the software code, assessment of the complexity of software designs remains elusive. Moreover, there are no clear guidelines to help software designers chose alternatives that reduce design complexity, improve design comprehensibility, and improve the maintainability of the software. This paper outlines a language independent approach to measuring software design complexity using objective and deterministic metrics. The paper outlines the metrics for two major software design notations; UML Class Diagrams and UML State Machines. The approach is based on the analysis of the design elements and their mutual interactions. The approach can be extended to cover other UML design notations.

Anomaly detection (AD) is very important across several real-world problems in the heavy industries and Internet-of-Things (IoT) domains. Traditional methods so far have categorized anomaly detection into (a) unsupervised, (b) semi-supervised and (c) supervised techniques. A relatively unexplored direction is the development of context aware anomaly detection systems which can build on top of any of these three techniques by using side information. Context can be captured from a different modality such as semantic graphs encoding grouping of sensors governed by the physics of the asset. Process flow diagrams of an operational plant depicting causal relationships between sensors can also provide useful context for ML algorithms. Capturing such semantics by itself can be pretty challenging, however, our paper mainly focuses on, (a) designing and implementing effective anomaly detection pipelines using sparse Gaussian Graphical Models with various statistical distance metrics, and (b) differentiating these pipelines by embedding contextual semantics inferred from graphs so as to obtain better KPIs in practice. The motivation for the latter of these two has been explained above, and the former in particular is well motivated by the relatively mediocre performance of highly parametric deep learning methods for small tabular datasets (compared to images) such as IoT sensor data. In contrast to such traditional automated deep learning (AutoAI) techniques, our anomaly detection system is based on developing semantics-driven industry specific ML pipelines which perform scalable computation evaluating several models to identify the best model. We benchmark our AD method against state-of-the-art AD techniques on publicly available UCI datasets. We also conduct a case study on IoT sensor and semantic data procured from a large thermal energy asset to evaluate the importance of semantics in enhancing our pipelines. In addition, we also provide explainable insights for our model which provide a complete perspective to a reliability engineer.

The electrical power system is the backbone of our nations critical infrastructure. It has been designed to withstand single component failures based on a set of reliability metrics which have proven acceptable during normal operating conditions. However, in recent years there has been an increasing frequency of extreme weather events. Many have resulted in widespread long-term power outages, proving reliability metrics do not provide adequate energy security. As a result, researchers have focused their efforts resilience metrics to ensure efficient operation of power systems during extreme events. A resilient system has the ability to resist, adapt, and recover from disruptions. Therefore, resilience has demonstrated itself as a promising concept for currently faced challenges in power distribution systems. In this work, we propose an operational resilience metric for modern power distribution systems. The metric is based on the aggregation of system assets adaptive capacity in real and reactive power. This metric gives information to the magnitude and duration of a disturbance the system can withstand. We demonstrate resilience metric in a case study under normal operation and during a power contingency on a microgrid. In the future, this information can be used by operators to make more informed decisions based on system resilience in an effort to prevent power outages.

Physical Unclonable Function is an innovative hardware security primitives that exploit the physical characteristics of a physical object to generate a unique identifier, which play the role of the object's fingerprint. Silicon PUF, a popular type of PUFs, exploits the variation in the manufacturing process of integrated circuits (ICs). It needs an input called challenge to generate the response as an output. In addition, of classical attacks, PUFs are vulnerable to physical and modeling attacks. The performance of the PUFs is measured by several metrics like reliability, uniqueness and uniformity. So as an evidence, the main goal is to provide a complete tool that checks the strength and quantifies the performance of a given physical unconscionable function. This paper provides a tool and develops a set of metrics that can achieve safely the proposed goal.

The numerous attacks on ICS systems have made severe threats to critical infrastructure. Extensive studies have focussed on the risk assessment of discovering vulnerabilities. However, to identify Zero-day vulnerabilities is challenging because they are unknown to defenders. Here we sought to measure ICS system zero-day risk by building an enhanced attack graph for expected attack path exploiting zero-day vulnerability. In this study, we define the security metrics of Zero-day vulnerability for an ICS. Then we created a Zero-day attack graph to guide how to harden the system by measuring attack paths that exploiting zero-day vulnerabilities. Our studies identify the vulnerability assessment method on ICS systems considering Zero-day Vulnerability by zero-day attack graph. Together, our work is essential to ICS systems security. By assessing unknown vulnerability risk to close the imbalance between attackers and defenders.

Over the years, a number of vulnerability scoring frameworks have been proposed to characterize the severity of known vulnerabilities in software-dependent systems. These frameworks provide security metrics to support decision-making in system development and security evaluation and assurance activities. When used in this context, it is imperative that these security metrics be sound, meaning that they can be consistently measured in a reproducible, objective, and unbiased fashion while providing contextually relevant, actionable information for decision makers. In this paper, we evaluate the soundness of the security metrics obtained via several vulnerability scoring frameworks. The evaluation is based on the Method for DesigningSound Security Metrics (MDSSM). We also present several recommendations to improve vulnerability scoring frameworks to yield more sound security metrics to support the development of secure software-dependent systems.

We focus on resilience towards covert attacks on Cyber-Physical Systems (CPS). We define the new k-steerability and l-monitorability control-theoretic concepts. k-steerability reflects the ability to act on every individual plant state variable with at least k different groups of functionally diverse input signals. l-monitorability indicates the ability to monitor every individual plant state variable with £ different groups of functionally diverse output signals. A CPS with k-steerability and l-monitorability is said to be (k, l)-resilient. k and l, when both greater than one, provide the capability to mitigate the impact of covert attacks when some signals, but not all, are compromised. We analyze the influence of k and l on the resilience of a system and the ability to recover its state when attacks are perpetrated. We argue that the values of k and l can be augmented by combining redundancy and diversity in hardware and software techniques that apply the moving target paradigm.

The importance of Cyber-Physical Systems (CPS) gains more and more weight in our daily business and private life. Although CPS build the backbone for major trends, like Industry 4.0 and connected vehicles, they also propose many new challenges. One major challenge can be found in achieving a high level of security within such highly connected environments, in which an unpredictable number of heterogeneous systems with often-distinctive characteristics interact with each other. In order to develop high-level security solutions, system designers must eventually know the current level of security of their specification. To this end, security metrics and scoring frameworks are essential, as they quantitatively express security of a given design or system. However, existing solutions may not be able to handle the proposed challenges of CPS, as they mainly focus on one particular system and one specific attack. Therefore, we aim to elaborate a security scoring mechanism, which can efficiently be used in CPS, while considering all essential information. We break down each system within the CPS into its core functional blocks and analyze a variety of attacks in terms of exploitability, scalability of attacks, as well as potential harm to targeted assets. With this approach, we get an overall assessment of security for the whole CPS, as it integrates the security-state of all interacting systems. This allows handling the presented complexity in CPS in a more efficient way, than existing solutions.

Embedded Systems (ES) development has been historically focused on functionality rather than security, and today it still applies in many sectors and applications. However, there is an increasing number of security threats over ES, and a successful attack could have economical, physical or even human consequences, since many of them are used to control critical applications. A standardized and general accepted security testing framework is needed to provide guidance, common reporting forms and the possibility to compare the results along the time. This can be achieved by introducing security metrics into the evaluation or assessment process. If carefully designed and chosen, metrics could provide a quantitative, repeatable and reproducible value that would reflect the level of security protection of the ES. This paper analyzes the features that a good security metric should exhibit, introduces a taxonomy for classifying them, and finally, it carries out a literature survey on security metrics for the security evaluation of ES. In this review, more than 500 metrics were collected and analyzed. Then, they were reduced to 169 metrics that have the potential to be applied to ES security evaluation. As expected, the 77.5% of them is related exclusively to software, and only the 0.6% of them addresses exclusively hardware security. This work aims to lay the foundations for constructing a security evaluation methodology that uses metrics so as to quantify the security level of an ES.

We consider a two-player zero-sum network routing game in which a router wants to maximize the amount of legitimate traffic that flows from a given source node to a destination node and an attacker wants to block as much legitimate traffic as possible by flooding the network with malicious traffic. We address scenarios with asymmetric information, in which the router must reveal its policy before the attacker decides how to distribute the malicious traffic among the network links, which is naturally modeled by the notion of Stackelberg equilibria. The paper focuses on parallel networks, and includes three main contributions: we show that computing the optimal attack policy against a given routing policy is an NP-hard problem; we establish conditions under which the Stackelberg equilibria lead to no regret; and we provide a metric that can be used to quantify how uncertainty about the attacker's capabilities limits the router's performance.

The Common Weakness Enumeration (CWE) is a prominent list of software weakness types. This list is used by vulnerability databases to describe the underlying security flaws within analyzed vulnerabilities. This linkage opens the possibility of using the analysis of software vulnerabilities to identify the most significant weaknesses that enable those vulnerabilities. We accomplish this through creating mashup views combining CWE weakness taxonomies with vulnerability analysis data. The resulting graphs have CWEs as nodes, edges derived from multiple CWE taxonomies, and nodes adorned with vulnerability analysis information (propagated from children to parents). Using these graphs, we develop a suite of metrics to identify the most significant weakness types (using the perspectives of frequency, impact, exploitability, and overall severity).

The usage of connected devices and their role within our daily- and business life gains more and more impact. In addition, various derivations of Cyber-Physical Systems (CPS) reach new business fields, like smart healthcare or Industry 4.0. Although these systems do bring many advantages for users by extending the overall functionality of existing systems, they come with several challenges, especially for system engineers and architects. One key challenge consists in achieving a sufficiently high level of security within the CPS environment, as sensitive data or safety-critical functions are often integral parts of CPS. Being system of systems (SoS), CPS complexity, unpredictability and heterogeneity complicate analyzing the overall level of security, as well as providing a way to detect ongoing attacks. Usually, security metrics and frameworks provide an effective tool to measure the level of security of a given component or system. Although several comprehensive surveys exist, an assessment of the effectiveness of the existing solutions for CPS environments is insufficiently investigated in literature. In this work, we address this gap by benchmarking a carefully selected variety of existing security metrics in terms of their usability for CPS. Accordingly, we pinpoint critical CPS challenges and qualitatively assess the effectiveness of the existing metrics for CPS systems.

The emergence of Industrial Cyber-Physical Systems (ICPS) in today's business world is still steadily progressing to new dimensions. Although they bring many new advantages to business processes and enable automation and a wider range of service capability, they also propose a variety of new challenges. One major challenge, which is introduced by such System-of-Systems (SoS), lies in the security aspect. As security may not have had that significant role in traditional embedded system engineering, a generic way to measure the level of security within an ICPS would provide a significant benefit for system engineers and involved stakeholders. Even though many security metrics and frameworks exist, most of them insufficiently consider an SoS context and the challenges of such environments. Therefore, we aim to define a security metric for ICPS, which measures the level of security during the system design, tests, and integration as well as at runtime. For this, we try to focus on a semantic point of view, which on one hand has not been considered in security metric definitions yet, and on the other hand allows us to handle the complexity of SoS architectures. Furthermore, our approach allows combining the critical characteristics of an ICPS, like uncertainty, required reliability, multi-criticality and safety aspects.

Software Quality Testing has always been a crucial part of the software development process and lately, there has been a rise in the usage of testing applications. While a well-planned and performed test, regardless of its nature - automated or manual, is a key factor when deciding on the results of the test, it is often not enough to give a more deep and thorough view of the whole process. That can be achieved with properly selected software metrics that can be used for proper risk assessment and evaluation of the development.This paper considers the most commonly used metrics when measuring a performed test and examines metrics that can be applied in the development process.

As networks continue to grow in complexity using wired and wireless technologies, efficient testing solutions should accommodate such changes and growth. Network simulators provide a network-independent environment to provide different types of network testing. This paper is motivated by the observation that, in many cases in the literature, the success of developed network protocols is very sensitive to the initial conditions and assumptions of the testing scenarios. Network services are deployed in complex environments; results of testing and simulation can vary from one environment to another and sometimes in the same environment at different times. Our goal is to propose mutation-based integration testing that can be deployed with network protocols and serve as Built-in Tests (BiT).This paper proposes an integrated mutation testing framework to achieve systematic test cases' generation for different scenario types. Scenario description and variables' setting should be consistent with the protocol specification and the simulation environment. We focused on creating test cases for critical scenarios rather than preliminary or simplified scenarios. This will help users to report confident simulation results and provide credible protocol analysis. The criticality is defined as a combination of network performance metrics and critical functions' coverage. The proposed solution is experimentally proved to obtain accurate evaluation results with less testing effort by generating high-quality testing scenarios. Generated test scenarios will serve as BiTs for the network simulator. The quality of the test scenarios is evaluated from three perspectives: (i) code coverage, (ii) mutation score and (iii) testing effort. In this work, we implemented the testing framework in NS2, but it can be extended to any other simulation environment.

Accessing the secured data through the network is a major task in emerging technology. Data needs to be protected from the network vulnerabilities, malicious users, hackers, sniffers, intruders. The novel framework has been designed to provide high security in data transaction through computer network. The implant of network amalgamation in the recent trends, make the way in security enhancement in an efficient manner through the machine learning algorithm. In this system the usage of the biometric authenticity plays a vital role for unique approach. The novel mathematical approach is used in machine learning algorithms to solve these problems and provide the security enhancement. The result shows that the novel method has consistent improvement in enhancing the security of data transactions in the emerging technologies.

The purpose of this paper is to analyze all Cloud based Service Models, Continuous Integration, Deployment and Delivery process and propose an Automated Continuous Testing and testing as a service based TestBot and metrics dashboard which will be integrated with all existing automation, bug logging, build management, configuration and test management tools. Recently cloud is being used by organizations to save time, money and efforts required to setup and maintain infrastructure and platform. Continuous Integration and Delivery is in practice nowadays within Agile methodology to give capability of multiple software releases on daily basis and ensuring all the development, test and Production environments could be synched up quickly. In such an agile environment there is need to ramp up testing tools and processes so that overall regression testing including functional, performance and security testing could be done along with build deployments at real time. To support this phenomenon, we researched on Continuous Testing and worked with industry professionals who are involved in architecting, developing and testing the software products. A lot of research has been done towards automating software testing so that testing of software product could be done quickly and overall testing process could be optimized. As part of this paper we have proposed ACT TestBot tool, metrics dashboard and coined 4S quality metrics term to quantify quality of the software product. ACT testbot and metrics dashboard will be integrated with Continuous Integration tools, Bug reporting tools, test management tools and Data Analytics tools to trigger automation scripts, continuously analyze application logs, open defects automatically and generate metrics reports. Defect pattern report will be created to support root cause analysis and to take preventive action.