Biblio

The autonomy of intelligent systems and their safety rely on their ability for local decision making based on collected environmental information. This is even more for cyber-physical systems running safety critical activities. While this intelligence is partial and fragmented, and cognitive techniques are of limited maturity, the decision function must produce results whose validity and scope must be weighted in light of the underlying assumptions, unavoidable uncertainty and hypothetical safety limitation. Besides the cognitive techniques dependability, it is about the assurance level of the decision self-making. Beyond the pure decision-making capabilities of the autonomous intelligent system, we need techniques that guarantee the system assurance required for the intended use. Security mechanisms for cognitive systems may be consequently tightly intricated. We propose a trustworthiness module which is part of the system and its resulting safety. In this paper, we briefly review the state of the art regarding the dependability of cognitive techniques, the assurance level definition in this context, and related engineering practices. We elaborate regarding the design of autonomous intelligent systems safety, then we discuss its security design and approaches for the mitigation of safety violations by the cognitive functions.

This work describes a top down systems security requirements analysis approach for understanding and eliciting security requirements for a notional small unmanned aerial system (SUAS). More specifically, the System-Theoretic Process Analysis approach for Security (STPA-Sec) is used to understand and elicit systems security requirements. The effort employs STPA-Sec on a notional SUAS system case study to detail the development of functional-level security requirements, design-level engineering considerations, and architectural-level security specification criteria early in the system life cycle when the solution trade-space is largest rather than merely examining components and adding protections during system operation or sustainment. These details were elaborated during a semester independent study research effort by two United States Air Force Academy Systems Engineering cadets, guided by their instructor and a series of working group sessions with UAS operators and subject matter experts. This work provides insight into a viable systems security requirements analysis approach which results in traceable security, safety, and resiliency requirements that can be designed-for, built-to, and verified with confidence.

Internet of Things (IoT) is experiencing significant growth in the safety-critical applications which have caused new security challenges. These devices are becoming targets for different types of physical attacks, which are exacerbated by their diversity and accessibility. Therefore, there is a strict necessity to support embedded software developers to identify and remediate the vulnerabilities and create resilient applications against such attacks. In this paper, we propose a hardware security vulnerability assessment based on fault injection of an embedded application. In our security assessment, we apply a fault injection attack by using our clock glitch generator on a critical medical IoT device. Furthermore, we analyze the potential risks of ignoring these attacks in this embedded application. The results will inform the embedded software developers of various security risks and the required steps to improve the security of similar MCU-based applications. Our hardware security assessment approach is easy to apply and can lead to secure embedded IoT applications against fault attacks.

Heterogeneous system-on-chip platforms with multiple processing cores are becoming increasingly common in safety-and security-critical embedded systems. To facilitate a logical isolation of physically connected on-chip components, internal communication links of such platforms are often equipped with dedicated access protection units. When performed manually, however, the configuration of these units can be both time-consuming and error-prone. To resolve this issue, we present a formal model and a corresponding design methodology that allows developers to specify access permissions and information flow requirements for embedded systems in a mostly platform-independent manner. As part of the methodology, the consistency between the permissions and the requirements is automatically verified and an extensible generation framework is used to transform the abstract permission declarations into configuration code for individual access protection units. We present a prototypical implementation of this approach and validate it by generating configuration code for the access protection unit of a commercially available multiprocessor system-on-chip.

Verifying complex Cyber-Physical Systems (CPS) is increasingly important given the push to deploy safety-critical autonomous features. Unfortunately, traditional verification methods do not scale to the complexity of these systems and do not provide systematic methods to protect verified properties when not all the components can be verified. To address these challenges, this paper proposes a real-time mixed-trust computing framework that combines verification and protection. The framework introduces a new task model, where an application task can have both an untrusted and a trusted part. The untrusted part allows complex computations supported by a full OS with a realtime scheduler running in a VM hosted by a trusted hypervisor. The trusted part is executed by another scheduler within the hypervisor and is thus protected from the untrusted part. If the untrusted part fails to finish by a specific time, the trusted part is activated to preserve safety (e.g., prevent a crash) including its timing guarantees. This framework is the first allowing the use of untrusted components for CPS critical functions while preserving logical and timing guarantees, even in the presence of malicious attackers. We present the framework design and implementation along with the schedulability analysis and the coordination protocol between the trusted and untrusted parts. We also present our Raspberry Pi 3 implementation along with experiments showing the behavior of the system under failures of untrusted components, and a drone application to demonstrate its practicality.

Data fusion, as a means to improve aircraft and air traffic safety, is a recent focus of some researchers and system developers. Increases in data volume and processing needs necessitate more powerful hardware and more flexible software architectures to satisfy these needs. Such improvements in processed data also mean the overall system becomes more complex and correspondingly, resulting in a potentially significantly larger cyber-attack space. Today's multicore processors are one means of satisfying the increased computational needs of data fusion-based systems. When coupled with a real-time operating system (RTOS) capable of flexible core and application scheduling, large cabinets of (power hungry) single-core processors may be avoided. The functional and assurance capabilities of such an RTOS can be critical elements in providing application isolation, constrained data flows, and restricted hardware access (including covert channel prevention) necessary to reduce the overall cyber-attack space. This paper examines fundamental considerations of a multiple independent levels of security (MILS) architecture when supported by a multicore-based real-time operating system. The paper draws upon assurance activities and functional properties associated with a previous Common Criteria evaluation assurance level (EAL) 6+ / High-Robustness Separation Kernel certification effort and contrast those with activities performed as part of a MILS multicore related project. The paper discusses key characteristics and functional capabilities necessary to achieve overall system security and safety. The paper defines architectural considerations essential for scheduling applications on a multicore processor to reduce security risks. For civil aircraft systems, the paper discusses the applicability of the security assurance and architecture configurations to system providers looking to increase their resilience to cyber threats.

Software vulnerabilities often remain hidden until an attacker exploits the weak/insecure code. Therefore, testing the software from a vulnerability discovery perspective becomes challenging for developers if they do not inspect their code thoroughly (which is time-consuming). We propose that vulnerability prediction using certain software metrics can support the testing process by identifying vulnerable code-components (e.g., functions, classes, etc.). Once a code-component is predicted as vulnerable, the developers can focus their testing efforts on it, thereby avoiding the time/effort required for testing the entire application. The current paper presents a study that compares how software metrics perform as vulnerability predictors for software projects developed in two different languages (Java vs Python). The goal of this research is to analyze the vulnerability prediction performance of software metrics for different programming languages. We designed and conducted experiments on security vulnerabilities reported for three Java projects (Apache Tomcat 6, Tomcat 7, Apache CXF) and two Python projects (Django and Keystone). In this paper, we focus on a specific type of code component: Functions. We apply Machine Learning models for predicting vulnerable functions. Overall results show that software metrics-based vulnerability prediction is more useful for Java projects than Python projects (i.e., software metrics when used as features were able to predict Java vulnerable functions with a higher recall and precision compared to Python vulnerable functions prediction).

Recent years have seen an increase in the data usage in cars, particularly as they become more autonomous and connected. With the rise in data use have come concerns about automotive cyber-security. An in-vehicle network shown to be particularly vulnerable is the Controller Area Network (CAN), which is the communication bus used by the car's safety critical and performance critical components. Cyber attacks on the CAN have been demonstrated, leading to research to develop attack detection and attack prevention systems. Such research requires representative attack demonstrations and data for testing. Obtaining this data is problematical due to the expense, danger and impracticality of using real cars on roads or tracks for example attacks. Whilst CAN simulators are available, these tend to be configured for testing conformance and functionality, rather than analysing security and cyber vulnerability. We therefore adapt a leading, industry-standard, CAN simulator to incorporate a core set of cyber attacks that are representative of those proposed by other researchers. Our adaptation allows the user to configure the attacks, and can be added easily to the free version of the simulator. Here we describe the simulator and, after reviewing the attacks that have been demonstrated and discussing their commonalities, we outline the attacks that we have incorporated into the simulator.

In this paper, we analyze the cyber resilience for the energy delivery systems (EDS) using critical system functionality (CSF). Some research works focus on identification of critical cyber components and services to address the resiliency for the EDS. Analysis based on the devices and services excluding the system behavior during an adverse event would provide partial analysis of cyber resilience. To address the gap, in this work, we utilize the vulnerability graph representation of EDS to compute the system functionality under adverse condition. We use network criticality metric to determine CSF. We estimate the criticality metric using graph Laplacian matrix and network performance after removing links (i.e., disabling control functions, or services). We model the resilience of the EDS using CSF, and system recovery curve. We also provide a comprehensive analysis of cyber resilience by determining the critical devices using TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) and AHP (Analytical Hierarchy Process) methods. We present use cases of EDS illustrating the way control functions and services in EDS map to the vulnerability graph model. The simulation results show that we can estimate the resilience metric using different types of graphs that may assist in making an informed decision about EDS resilience.

Security model is an important subject in the field of low energy independence complexity theory. It takes security strategy as the core, changes the system from static protection to dynamic protection, and provides the basis for the rapid response of the system. A large number of empirical studies have been conducted to verify the cache consistency. The development of object oriented language is pure object oriented language, and the other is mixed object oriented language, that is, adding class, inheritance and other elements in process language and other languages. This paper studies a new object-oriented language application, namely GUT for a write-back cache, which is based on the study of simulation algorithm to solve all these challenges in the field of low energy independence complexity theory.

Functional Safety standards like ISO 26262 require a detailed analysis of the dependability of components subjected to perturbations. Radiation testing or even much more abstract RTL fault injection campaigns are costly and complex to set up especially for SoCs and Cyber Physical Systems (CPSs) comprising intertwined hardware and software. Moreover, some approaches are only applicable at the very end of the development cycle, making potential iterations difficult when market pressure and cost reduction are paramount. In this tutorial, we present a summary of classical state-of-the-art approaches, then alternative approaches for the dependability analysis that can give an early yet accurate estimation of the safety or security characteristics of HW-SW systems. Designers can rely on these tools to identify issues in their design to be addressed by protection mechanisms, ensuring that system dependability constraints are met with limited risk when subjected later to usual fault injections and to e.g., radiation testing or laser attacks for certification.

Emerging intelligent systems have stringent constraints including cost and power consumption. When they are used in critical applications, resiliency becomes another key requirement. Much research into techniques for fault tolerance and dependability has been successfully applied to highly critical systems, such as those used in space, where cost is not an overriding constraint. Further, most resiliency techniques were focused on dealing with failures in the hardware and bugs in the software. The next generation of systems used in critical applications will also have to be tolerant to test escapes after manufacturing, soft errors and transients in the electronics, hardware bugs, hardware and software Trojans and viruses, as well as intrusions and other security attacks during operation. This paper will assess the impact of these threats on the results produced by a critical system, and proposed solutions to each of them. It is argued that run-time checks at the application-level are necessary to deal with errors in the results.

Modern automotive embedded software is mostly designed using model-based design tools such as Simulink or SCADE, and source code is generated automatically from the models. Formal proof using symbolic model checking has been integrated in these tools and can provide a higher assurance by proving safety-critical properties. Our experience shows that proving properties involving time is rather challenging when they involve long durations and timers. These properties are generally not inductive and even advanced techniques such as PDR/IC3 are unable to handle them on production models in reasonable time. In this paper, we first present our industrial use case and comment on the results obtained with the existing model checkers. Then we present our invariant generator and methodology for selecting invariants according to physical dimensions. They enable the proof of properties with long-running timers. Finally, we discuss their implementation and benchmarks.

From financial services platforms to social networks to vehicle control, software has come to mediate many activities of daily life. Governing bodies and standards organizations have responded to this trend by creating regulations and standards to address issues such as safety, security and privacy. In this environment, the compliance of software development to standards and regulations has emerged as a key requirement. Compliance claims and arguments are often captured in assurance cases, with linked evidence of compliance. Evidence can come from testcases, verification proofs, human judgement, or a combination of these. That is, we try to build (safety-critical) systems carefully according to well justified methods and articulate these justifications in an assurance case that is ultimately judged by a human. Yet software is deeply rooted in uncertainty making pragmatic assurance more inductive than deductive: most of complex open-world functionality is either not completely specifiable (due to uncertainty) or it is not cost-effective to do so, and deductive verification cannot happen without specification. Inductive assurance, achieved by sampling or testing, is easier but generalization from finite set of examples cannot be formally justified. And of course the recent popularity of constructing software via machine learning only worsens the problem - rather than being specified by predefined requirements, machine-learned components learn existing patterns from the available training data, and make predictions for unseen data when deployed. On the surface, this ability is extremely useful for hard-to specify concepts, e.g., the definition of a pedestrian in a pedestrian detection component of a vehicle. On the other, safety assessment and assurance of such components becomes very challenging. In this talk, I focus on two specific approaches to arguing about safety and security of software under uncertainty. The first one is a framework for managing uncertainty in assurance cases (for "conventional" and "machine-learned" systems) by systematically identifying, assessing and addressing it. The second is recent work on supporting development of requirements for machine-learned components in safety-critical domains.

Companies develop their software in versions and iterations. Ensuring the security of each additional version using code review is costly and time consuming. This paper investigates automated tracing of the impacts of code changes on the security of a given software. To this end, we use call graphs to model the software code, and security assurance cases to model the security requirements of the software. Then we relate assurance case elements to code through the entry point methods of the software, creating a map of monitored security functions. This mapping allows to evaluate the security requirements that are affected by code changes. The approach is implemented in a set of tools and evaluated using three open-source ERP/E-commerce software applications. The limited evaluation showed that the approach is effective in identifying the impacts of code changes on the security of the software. The approach promises to considerably reduce the security assessment time of the subsequent releases and iterations of software, keeping the initial security state throughout the software lifetime.

Github Gist is a service provided by Github which is used by developers to share code snippets. While sharing, developers may inadvertently introduce security smells in code snippets as well, such as hard-coded passwords. Security smells are recurrent coding patterns that are indicative of security weaknesses, which could potentially lead to security breaches. The goal of this paper is to help software practitioners avoid insecure coding practices through an empirical study of security smells in publicly-available GitHub Gists. Through static analysis, we found 13 types of security smells with 4,403 occurrences in 5,822 publicly-available Python Gists. 1,817 of those Gists, which is around 31%, have at least one security smell including 689 instances of hard-coded secrets. We also found no significance relation between the presence of these security smells and the reputation of the Gist author. Based on our findings, we advocate for increased awareness and rigorous code review efforts related to software security for Github Gists so that propagation of insecure coding practices are mitigated.

Embedded systems that communicate with each other over the internet and build up a larger, loosely coupled (hardware) system with an unknown configuration at runtime is often referred to as a cyberphysical system. Many of these systems can become, due to its associated risks during their operation, safety critical. With increased complexity of such systems, the number of configurations can either be infinite or even unknown at design time. Hence, a certification at design time for such systems that documents a safe interaction for all possible configurations of all participants at runtime can become unfeasible. If such systems come together in a new configuration, a mechanism is required that can decide whether or not it is safe for them to interact. Such a mechanism can generally not be part of such systems for the sake of trust. Therefore, we present in the following sections the SEnSE device, short for Secure and Safe Embedded, that tackles these challenges and provides a secure and safe integration of safety-critical embedded systems.

Hybrid attack graphs are a powerful tool when analyzing the cybersecurity of a cyber-physical system. However, it is important to ensure that this tool correctly models reality, particularly when modelling safety-critical applications, such as a nuclear reactor. By automatically verifying that a simulation reaches the state predicted by an attack graph by analyzing the final state of the simulation, this verification procedure can be accomplished. As such, a mechanism to estimate if a simulation reaches the expected state in a hybrid attack graph is proposed here for the nuclear reactor domain.

While vehicle to everything (V2X) communication enables safety-critical automotive control systems to better support various connected services to improve safety and convenience of drivers, they also allow automotive attack surfaces to increase dynamically in modern vehicles. Many researchers as well as hackers have already demonstrated that they can take remote control of the targeted car by exploiting the vulnerabilities of in-vehicle networks such as Controller Area Networks (CANs). For assuring CAN security, we focus on how to authenticate electronic control units (ECUs) in real-time by addressing the security challenges of in-vehicle networks. In this paper, we propose a novel and lightweight authentication protocol with an attack-resilient tree algorithm, which is based on one-way hash chain. The protocol can be easily deployed in CAN by performing a firmware update of ECU. We have shown analytically that the protocol achieves a high level of security. In addition, the performance of the proposed protocol is validated on CANoe simulator for virtual ECUs and Freescale S12XF used in real vehicles. The results show that our protocol is more efficient than other authentication protocol in terms of authentication time, response time, and service delay.

Summary form only given. Symbolic execution is a systematic program analysis technique which explores multiple program behaviors all at once by collecting and solving symbolic path conditions over program paths. The technique has been recently extended with probabilistic reasoning. This approach computes the conditions to reach target program events of interest and uses model counting to quantify the fraction of the input domain satisfying these conditions thus computing the probability of event occurrence. This probabilistic information can be used for example to compute the reliability of an aircraft controller under different wind conditions (modeled probabilistically) or to quantify the leakage of sensitive data in a software system, using information theory metrics such as Shannon entropy. In this talk we review recent advances in symbolic execution and probabilistic reasoning and we discuss how they can be used to ensure the safety and security of software systems.

PHP is one of the most popular web development tools in use today. A major concern though is the improper and insecure uses of the language by application developers, motivating the development of various static analyses that detect security vulnerabilities in PHP programs. However, many of these approaches do not handle recent, important PHP features such as object orientation, which greatly limits the use of such approaches in practice. In this paper, we present OOPIXY, a security analysis tool that extends the PHP security analyzer PIXY to support reasoning about object-oriented features in PHP applications. Our empirical evaluation shows that OOPIXY detects 88% of security vulnerabilities found in micro benchmarks. When used on real-world PHP applications, OOPIXY detects security vulnerabilities that could not be detected using state-of-the-art tools, retaining a high level of precision. We have contacted the maintainers of those applications, and two applications' development teams verified the correctness of our findings. They are currently working on fixing the bugs that lead to those vulnerabilities.

Artificial software diversity is an effective way to prevent software vulnerabilities and errors to be exploited in code-reuse attacks. This is achieved by lowering the individual probability of a successful attack to a level that makes the attack unfeasible. Unfortunately, the existing approaches are not applicable to safety-critical real-time systems as they induce unacceptable performance overheads, they violate safety and timing guarantees, or they assume hardware resources which are typically not available in embedded systems. To overcome these problems, we propose a safe diversity approach that preserves the timing properties of real-time processes by controlling its impact on the worst case execution time (WCET). Our main idea is to use block-level diversity with a large, but fixed set of movable instruction sequences, and to use static WCET analysis to identify non-critical areas of code where it can safely be split into more movable instruction sequences.

The software supply chain is a source of cybersecurity risk for many commercial and government organizations. Public data may be used to inform automated tools for detecting software supply chain risk during continuous integration and deployment. We link data from the National Vulnerability Database (NVD) with open version control data for the open source project OpenSSL, a widely used secure networking library that made the news when a significant vulnerability, Heartbleed, was discovered in 2014. We apply the Alhazmi-Malaiya Logistic (AML) model for software vulnerability discovery to this case. This model predicts a sigmoid cumulative vulnerability discovery function over time. Some versions of OpenSSL do not conform to the predictions of the model because they contain a temporary plateau in the cumulative vulnerability discovery plot. This temporary plateau feature is an empirical signature of a security failure mode that may be useful in future studies of software supply chain risk.

Complex safety-critical devices require dependable communication. Dependability includes confidentiality and integrity as much as safety. Encrypting gateways with demilitarized zones, Multiple Independent Levels of Security architectures and the infamous Air Gap are diverse integration patterns for safety-critical infrastructure. Though resource restricted embedded safety devices still lack simple, certifiable, and efficient cryptography implementations. Following the recommended formal methods approach for safety-critical devices, we have implemented proven cryptography algorithms in the qualified model based language Scade as the Safety Leveraged Implementation of Data Encryption (SLIDE) library. Optimization for the synchronous dataflow language is discussed in the paper. The implementation for public-key based encryption and authentication is evaluated for real-world performance. The feasibility is shown by execution time benchmarks on an industrial safety microcontroller platform running a train control safety application.

Creating and implementing fault-tolerant distributed algorithms is a challenging task in highly safety-critical industries. Using formal methods supports design and development of complex algorithms. However, formal methods are often perceived as an unjustifiable overhead. This paper presents the experience and insights when using TLA+ and PlusCal to model and develop fault-tolerant and safety-critical modules for TAS Control Platform, a platform for railway control applications up to safety integrity level (SIL) 4. We show how formal methods helped us improve the correctness of the algorithms, improved development efficiency and how part of the gap between model and implementation has been closed by translation to C code. Additionally, we describe how we gained trust in the formal model and tools by following a specific design process called property-driven design, which also implicitly addresses software quality metrics such as code coverage metrics.