Biblio

Autonomous vehicles (AVs) are capable of making driving decisions autonomously using multiple sensors and a complex autonomous driving (AD) software. However, AVs introduce numerous unique security challenges that have the potential to create safety consequences on the road. Security mechanisms require a benchmark suite and an evaluation framework to generate comparable results. Unfortunately, AVs lack a proper benchmarking framework to evaluate the attack and defense mechanisms and quantify the safety measures. This paper introduces BenchAV – a security benchmark suite and evaluation framework for AVs to address current limitations and pressing challenges of AD security. The benchmark suite contains 12 security and performance metrics, and an evaluation framework that automates the metric collection process using Carla simulator and Robot Operating System (ROS).

Nowadays, automotive products are intelligence intensive and thus inevitably handle multiple functionalities under the current high-speed networking environment. The embedded virtualization has high potentials in the automotive industry, thanks to its advantages in function integration, resource utilization, and security. The invention of ARM virtualization extensions has made it possible to run open-source hypervisors, such as Xen and KVM, for embedded applications. Nevertheless, there is little work to investigate the performance of these hypervisors on automotive platforms. This paper presents a detailed analysis of different types of open-source hypervisors that can be applied in the ARM platform. We carry out the virtualization performance experiment from the perspectives of CPU, memory, file I/O, and some OS operation performance on Xen and Jailhouse. A series of microbenchmark programs have been designed, specifically to evaluate the real-time performance of various hypervisors and the relevant overhead. Compared with Xen, Jailhouse has better latency performance, stable latency, and little interference jitter. The performance experiment results help us summarize the advantages and disadvantages of these hypervisors in automotive applications.

Due to the decentralization, irreversibility, and traceability, blockchain has attracted significant attention and has been deployed in many critical industries such as banking and logistics. However, the micro-architecture characteristics of blockchain programs still remain unclear. What's worse, the large number of micro-architecture events make understanding the characteristics extremely difficult. We even lack a systematic approach to identify the important events to focus on. In this paper, we propose a novel benchmarking methodology dubbed BBS to characterize blockchain programs at micro-architecture level. The key is to leverage fuzzy set theory to identify important micro-architecture events after the significance of them is quantified by a machine learning based approach. The important events for single programs are employed to characterize the programs while the common important events for multiple programs form an importance vector which is used to measure the similarity between benchmarks. We leverage BBS to characterize seven and six benchmarks from Blockbench and Caliper, respectively. The results show that BBS can reveal interesting findings. Moreover, by leveraging the importance characterization results, we improve that the transaction throughput of Smallbank from Fabric by 70% while reduce the transaction latency by 55%. In addition, we find that three of seven and two of six benchmarks from Blockbench and Caliper are redundant, respectively.

Benchmarking is an important measure for companies to investigate their performance and to increase efficiency. As companies usually are reluctant to provide their key performance indicators (KPIs) for public benchmarks, privacy-preserving benchmarking systems are required. In this paper, we present an enhanced privacy-preserving benchmarking protocol, which we implemented and evaluated based on the real-world scenario of product cost optimisation. It is based on homomorphic encryption and enables cloud-based KPI comparison, providing a variety of statistical measures. The theoretical and empirical evaluation of our benchmarking system underlines its practicability.

Microservice architectures are steadily gaining adoption in industrial practice. At the same time, performance and resilience are important properties that need to be ensured. Even though approaches for performance and resilience have been developed (e.g., for anomaly detection and fault tolerance), there are no benchmarking environments for their evaluation under controlled conditions. In this paper, we propose a generative platform for benchmarking performance and resilience engineering approaches in microservice architectures, comprising an underlying metamodel, a generation platform, and supporting services for workload generation, problem injection, and monitoring.

The advancement in technology has changed how people work and what software and hardware people use. From conventional personal computer to GPU, hardware technology and capability have dramatically improved so does the operating systems that come along. Unfortunately, current industry practice to compare OS is performed with single perspective. It is either benchmark the hardware level performance or performs penetration testing to check the security features of an OS. This rigid method of benchmarking does not really reflect the true performance of an OS as the performance analysis is not comprehensive and conclusive. To illustrate this deficiency, the study performed hardware level and operational level benchmarking on Windows XP, Windows 7 and Windows 8 and the results indicate that there are instances where Windows XP excels over its newer counterparts. Overall, the research shows Windows 8 is a superior OS in comparison to its predecessors running on the same hardware. Furthermore, the findings also show that the automated benchmarking tools are proved less efficient benchmark systems that run on Windows XP and older OS as they do not support DirectX 11 and other advanced features that the hardware supports. There lies the need to have a unified benchmarking approach to compare other aspects of OS such as user oriented tasks and security parameters to provide a complete comparison. Therefore, this paper is proposing a unified approach for Operating System (OS) comparisons with the help of a Windows OS case study. This unified approach includes comparison of OS from three aspects which are; hardware level, operational level performance and security tests.

Continuously improving security on an information system requires unique combination of human aspect, policies, and technology. This acts as leverage for designing an access control management approach which avails only relevant parts of a system according to an end-users' scope of work. This paper introduces a framework for information security fundamentals at organizational and theoretical levels, to identify critical success factors that are vital in assessing an organization's security maturity through a model referred to as “information security digital divide maturity framework”. The foregoing is based on a developed conceptual framework for information security digital divide. The framework strives to divide system end-users into “specific information haves and have-nots”. It intends to assist organizations to continually evaluate and improve on their security governance, standards, and policies which permit access on the basis of each end-user's work scope. The framework was tested through two surveys targeting 90 end-users and 35 security experts.