Biblio

A new round of scientific and technological revolution and industrial reform promote the intelligent development of automobile and promote the deep integration of automobile with Internet, big data, communication and other industries. At the same time, it also brings network and data security problems to automobile, which is very easy to cause national security and social security risks. Intelligent vehicle Ethernet intrusion detection can effectively alleviate the security risk of vehicle network, but the complex attack means and vehicle compatibility have not been effectively solved. This research takes the vehicle Ethernet as the research object, constructs the machine learning samples for neural network, applies the self coding network technology combined with the original characteristics to the network intrusion detection algorithm, and studies a self-learning vehicle Ethernet intrusion detection algorithm. Through the application and test of vehicle terminal, the algorithm generated in this study can be used for vehicle terminal with Ethernet communication function, and can effectively resist 34 kinds of network attacks in four categories. This method effectively improves the network security defense capability of vehicle Ethernet, provides technical support for the network security of intelligent vehicles, and can be widely used in mass-produced intelligent vehicles with Ethernet.

The Internet of vehicles is emerging as an exciting application to improve safety and providing better services in the form of active road signs, pay-as-you-go insurance, electronic toll, and fleet management. Internet connected vehicles are exposed to new attack vectors in the form of cyber threats and with the increasing trend of cyber attacks, the success of autonomous vehicles depends on their security. Existing techniques for IoV security are based on the un-realistic assumption that all the vehicles are equipped with the same hardware (at least in terms of computational capabilities). However, the hardware platforms used by various vehicle manufacturers are highly heterogeneous. Therefore, a security protocol designed for IoVs should be able to detect the computational capabilities of the underlying platform and adjust the security primitives accordingly. To solve this issue, this paper presents a technique for algorithm-hardware separation for IoV security. The proposed technique uses an iterative routine and the corresponding execution time to detect the computational capabilities of a hardware platform using an artificial intelligence based inference engine. The results on three different commonly used micro-controllers show that the proposed technique can effectively detect the type of hardware platform with up to 100% accuracy.

With the gradually popular high-speed wireless networks and 5G environments, the quality and reliability of network services will be suited for mobile vehicles. In addition to communicating information between vehicles, they can also communicate information with surrounding roadside equipment, pedestrians or traffic signs, and thus improve the road safety of passers-by.Recently, various countries have continuously invested in research on autonomous driving and unmanned vehicles. The open communication environment of the Internet of Vehicles in 5G will expose all personal information in the field of wireless networks. This research is based on the consideration of information security and personal data protection. We will focus on how to protect the real-time transmission of information between mobile vehicles to prevent from imbedding or altering important transmission information by unauthorized vehicles, drivers or passers-by participating in communications. Moreover, this research proposes a multi-level security key management agreement based on a dynamic M-tree structure for Internet of Vehicles to achieve flexible and scalable key management on large-scale Internet of Vehicles.

With advantages of low cost, high mobility, and flexible deployment, unmanned aerial vehicle (UAVs) are employed to efficiently detect abnormal vehicle behaviors (AVBs) in the internet of vehicles (IoVs). However, due to limited resources including battery, computing, and communication, UAVs are selfish to work cooperatively. To solve the above problem, in this paper, a game theoretical UAV incentive scheme in IoVs is proposed. Specifically, the abnormal behavior model is first constructed, where three model categories are defined: velocity abnormality, distance abnormality, and overtaking abnormality. Then, the barging pricing framework is designed to model the interactions between UAVs and IoVs, where the transaction prices are determined with the abnormal behavior category detected by UAVs. At last, simulations are conducted to verify the feasibility and effectiveness of our proposed scheme.

The transportation system is becoming tremendously important in today's human activities and the number of urban vehicles grows rapidly. The vehicle theft also has become a shared concern for all vehicle owners. However, the present anti-theft system which maybe high reliable, lack of proper mechanism for preventing theft before it happens. This work proposes the internet of things based smart vehicle security staring system; efficient security provided to the vehicle owners relies on securing car ignition system by using a developed android application running on smart phone connected to the designed system installed in vehicle. With this system it is non- viable to access the vehicle's functional system in case the ignition key has been stolen or lost. It gives the drivers the ability to stay connected with their vehicle. Whenever the ignition key is stolen or lost, it is impossible to start the vehicle as the ignition system is still locked on the vehicle start and only the authorized person will be able to start the vehicle at convenient time with the combination of ignition key and smart phone application. This study proposes to design the system that uses node MCU, Bluetooth low energy (BLE), transistors, power relays and android smartphone in system testing. In addition, it is cost effective and once installed in the vehicle there is no more cost of maintenance.

We propose a human-operator guided planning approach to pushing-based manipulation in clutter. Most recent approaches to manipulation in clutter employs randomized planning. The problem, however, remains a challenging one where the planning times are still in the order of tens of seconds or minutes, and the success rates are low for difficult instances of the problem. We build on these control-based randomized planning approaches, but we investigate using them in conjunction with human-operator input. In our framework, the human operator supplies a high-level plan, in the form of an ordered sequence of objects and their approximate goal positions. We present experiments in simulation and on a real robotic setup, where we compare the success rate and planning times of our human-in-the-loop approach with fully autonomous sampling-based planners. We show that with a minimal amount of human input, the low-level planner can solve the problem faster and with higher success rates.

The present study proposes a method for detecting objects with a high recall rate for human-supported video annotation. In recent years, automatic annotation techniques such as object detection and tracking have become more powerful; however, detection and tracking of occluded objects, small objects, and blurred objects are still difficult. In order to annotate such objects, manual annotation is inevitably required. For this reason, we envision a human-supported video annotation framework in which over-detected objects (i.e., false positives) are allowed to minimize oversight (i.e., false negatives) in automatic annotation and then the over-detected objects are removed manually. This study attempts to achieve human-in-the-loop object detection with an emphasis on suppressing the oversight for the former stage of processing in the aforementioned annotation framework: bi-directional deep SORT is proposed to reliably capture missed objects and annotation-free segment identification (AFSID) is proposed to identify video frames in which manual annotation is not required. These methods are reinforced each other, yielding an increase in the detection rate while reducing the burden of human intervention. Experimental comparisons using a pedestrian video dataset demonstrated that bi-directional deep SORT with AFSID was successful in capturing object candidates with a higher recall rate over the existing deep SORT while reducing the cost of manpower compared to manual annotation at regular intervals.

We present a scalable architecture based on a trained filter bank for input pre-processing and a recurrent neural network (RNN) for the detection of atrial fibrillation in electrocardiogram (ECG) signals, with the focus on enabling a very efficient hardware implementation as application-specific integrated circuit (ASIC). Our already very efficient base architecture is further improved by replacing the RNN with a delta-encoded gated recurrent unit (GRU) and adding a confidence measure (CM) for terminating the computation as early as possible. With these optimizations, we demonstrate a reduction of the processing load of 58 % on an internal dataset while still achieving near state-of-the-art classification results on the Physionet ECG dataset with only 1202 parameters.

Fulcrum decoders can trade off between computational complexity and the number of received packets. This allows heterogeneous nodes to decode at different level of complexity in accordance with their computing power. Variations of Fulcrum codes, like dynamic sparsity and expansion packets (DSEP) have significantly reduced the encoders and decoders' complexity by using dynamic sparsity and expansion packets. However, limited effort had been done for recoders of Fulcrum codes and their variations, limiting their full potential when being deployed at multi-hop networks. In this paper, we investigate the drawback of the conventional Fulcrum recoding and introduce a novel recoding scheme for the family of Fulcrum codes by limiting the buffer size, and thus memory needs. Our evaluations indicate that DSEP recoding mechamism increases the recoding goodput by 50%, and reduces the decoding overhead by 60%-90% while maintaining high decoding goodput at receivers and small memory usage at recoders compared with the conventional Fulcrum recoding. This further reduces the resources needed for Fulcrum codes at the recoders.

The full integration of Remotely Piloted Aircraft Systems (RPAS) in non-segregated airspace is one of the major objectives for the worldwide aviation organizations and authorities. However, there are several technological and regulatory issues due to the increase of the air traffic in the next years and to the need of keeping high safety levels. In this framework, a real-time validation environment capable to simulate complex scenarios related to future air traffic management (ATM) conditions is of paramount importance. These facilities allow detailed testing and tuning of new technologies and procedures before executing flight tests. With such motivations, the Italian Aerospace Research Centre has developed the Integrated Simulation Facility (ISF) able to accurately reproduce ATM complex scenarios in real-time with hardware and human in-the-loop simulations, aiming to validate new ATM procedures and innovative system prototypes for RPAS and General Aviation aircraft. In the present work, the ISF facility has been used for reproducing relevant ATM scenarios to validate the functionalities of a Detect and Avoid system (DAA). The results of the ISF test campaign demonstrate the effectiveness of the developed algorithm in the autonomous resolution of mid-air collisions in presence of both air traffic and fixed obstacles (i.e. bad weather areas, no-fly-zone and terrain) and during critical flight phases, thus exceeding the current DAA state-of-the-art.

One of the typical autonomous driving systems is a human-machine cooperative system that intervenes in the driver operation. The autonomous driving needs to make consideration of the driver individuality in addition to safety. This paper considers a human-machine cooperative system balancing safety with the driver individuality using the Human-In-The-Loop System (HITLS) for rear-wheel steering control. This paper assumes that it is safe for HITLS to follow the target side-slip angle and target angular velocity without conflicts between the controller and driver operations. We propose HITLS using the primal-dual algorithm and the internal model control (IMC) type I-PD controller. In HITLS, the signal expander delimits the human-selectable operating range and the controller cooperates stably the human operation and automated control in that range. The primal-dual algorithm realizes the driver and the signal expander. Our outcomes are the making of the rear-wheel steering system which converges to the target value while reflecting the driver individuality.

In this paper, we propose a method of stochastic model predictive control for energy management systems including human-in-the-loop. Here, we consider an air-conditioning system consisting of some rooms. Human decision making about the set temperature is modeled by a discrete-time Markov chain. The finite-time optimal control problem solved in the controller is reduced to a mixed integer linear programming problem.

How to develop AI systems that can explain how they made decisions is one of the important and hot topics today. Inspired by the dual-process theory in psychology, this paper proposes a human-in-the-loop approach to develop System-2 AI that makes an inference logically and outputs interpretable explanation. Our proposed method first asks crowd workers to raise understandable features of objects of multiple classes and collect training data from the Internet to generate classifiers for the features. Logical decision rules with the set of generated classifiers can explain why each object is of a particular class. In our preliminary experiment, we applied our method to an image classification of Asian national flags and examined the effectiveness and issues of our method. In our future studies, we plan to combine the System-2 AI with System-1 AI (e.g., neural networks) to efficiently output decisions.

Human-in-the-Loop has been receiving special attention from the data science and machine learning community. It is essential to realize the advantages of human feedback and the pressing need for manual annotation to improve machine learning performance. Recent advancements in natural language processing (NLP) and machine learning have created unique challenges and opportunities for digital humanities research. In particular, there are ample opportunities for NLP and machine learning researchers to analyze data from literary texts and use these complex source texts to broaden our understanding of human sentiment using the human-in-the-loop approach. This paper presents our understanding of how human annotators differ from machine annotators in sentiment analysis tasks and how these differences can contribute to designing systems for the "human in the loop" sentiment analysis in complex, unstructured texts. We further explore the challenges and benefits of the human-machine collaboration for sentiment analysis using a case study in Greek tragedy and address some open questions about collaborative annotation for sentiments in literary texts. We focus primarily on (i) an analysis of the challenges in sentiment analysis tasks for humans and machines, and (ii) whether consistent annotation results are generated from multiple human annotators and multiple machine annotators. For human annotators, we have used a survey-based approach with about 60 college students. We have selected six popular sentiment analysis tools for machine annotators, including VADER, CoreNLP's sentiment annotator, TextBlob, LIME, Glove+LSTM, and RoBERTa. We have conducted a qualitative and quantitative evaluation with the human-in-the-loop approach and confirmed our observations on sentiment tasks using the Greek tragedy case study.

Distant supervision is a widely used data labeling method for relation extraction. While aligning knowledge base with the corpus, distant supervision leads to a mass of wrong labels which are defined as noise. The pattern-based denoising model has achieved great progress in selecting trustable sentences (instances). However, the writing of relation-specific patterns heavily relies on expert’s knowledge and is a high labor intensity work. To solve these problems, we propose a noise reduction framework, NOIR, to iteratively select trustable sentences with a little help of a human. Under the guidance of experts, the iterative process can avoid semantic drift. Besides, NOIR can help experts discover relation-specific tokens that are hard to think of. Experimental results on three real-world datasets show the effectiveness of the proposed method compared with state-of-the-art methods.

In this work, a new human-computer assistive technology gadget designed for people with impairments is evaluated. The developed human-in-the-loop interface device has an embedded assistance controller and can replace the traditional mouse, gamepad and keyboard, enabling human-computer hands-free full access. This work is concerned with the assistive device performance characterization aspects. Based on the experiments carried out, the human-computer performance improvement with the embedded controller is analysed in detail. Results show that adding the human-in-the-loop assistance controller improves human-computer hands-free skills, which is an innovative contribution for the replacement of computer interfaces that depend on the human hands.

One of the biggest problems in the current society is people's safety. Safety measures and mechanisms are especially important in the case of vulnerable social groups, such as migrants, homeless, and victims of domestic and/or sexual violence. In order to cope with this problem, we witness an increasing number of personal alarm systems in the market, most of them based on panic buttons. Nevertheless, none of them has got widespread acceptance mainly because of limited Human-Computer Interaction. In the context of this work, we developed an innovative mobile application that recognizes an attack through speech and gesture recognition. This paper describes such a system and presents its features, some of them based on the emerging concept of Human-in-the-Loop Cyber-physical Systems and new concepts of Human-Computer Interaction.

In this study, we present the abstracted pedestrian behavior representation and abnormal detection method based on user feedback for pedestrian video surveillance system. Video surveillance data is large in size and difficult to process in real time. To solve this problem, we suggested a method of expressing the pedestrian behavior with abbreviated map. In the video surveillance system, false detection of an abnormal situation becomes a big problem. If surveillance user can guide the false detection case as human in the loop, the surveillance system can learn the case and reduce the false detection error in the future. We suggested user feedback based abnormal pedestrian detection method. By the suggested user feedback algorithm, the false detection can be reduced to less than 0.5%.

Deep learning has undergone tremendous advancements in computer vision studies. The training of deep learning neural networks depends on a considerable amount of ground truth datasets. However, labeling ground truth data is a labor-intensive task, particularly for large-volume video analytics applications such as video surveillance and vehicles detection for autonomous driving. This paper presents a rapid and accurate method for associative searching in big image data obtained from security monitoring systems. We developed a semi-automatic moving object annotation method for improving deep learning models. The proposed method comprises three stages, namely automatic foreground object extraction, object annotation in subsequent video frames, and dataset construction using human-in-the-loop quick selection. Furthermore, the proposed method expedites dataset collection and ground truth annotation processes. In contrast to data augmentation and data generative models, the proposed method produces a large amount of real data, which may facilitate training results and avoid adverse effects engendered by artifactual data. We applied the constructed annotation dataset to train a deep learning you-only-look-once (YOLO) model to perform vehicle detection on street intersection surveillance videos. Experimental results demonstrated that the accurate detection performance was improved from a mean average precision (mAP) of 83.99 to 88.03.

This research concerns the detection of unauthorised access within hospital networks through the real-time analysis of audit logs. Privacy is a primary concern amongst patients due to the rising adoption of Electronic Patient Record (EPR) systems. There is growing evidence to suggest that patients may withhold information from healthcare providers due to lack of Trust in the security of EPRs. Yet, patient record data must be available to healthcare providers at the point of care. Ensuring privacy and confidentiality of that data is challenging. Roles within healthcare organisations are dynamic and relying on access control is not sufficient. Through proactive monitoring of audit logs, unauthorised accesses can be detected and presented to an analyst for review. Advanced data analytics and visualisation techniques can be used to aid the analysis of big data within EPR audit logs to identify and highlight pertinent data points. Employing a human-in-the-loop model ensures that suspicious activity is appropriately investigated and the data analytics is continuously improving. This paper presents a system that employs a Human-in-the-Loop Machine Learning (HILML) algorithm, in addition to a density-based local outlier detection model. The system is able to detect 145 anomalous behaviours in an unlabelled dataset of 1,007,727 audit logs. This equates to 0.014% of the EPR accesses being labelled as anomalous in a specialist Liverpool (UK) hospital.

Among the operational shortfalls that hinder law enforcement from achieving greater success in preventing terrorist attacks is the difficulty in dynamically assessing individualized violent extremism risk at scale given the enormous amount of primarily text-based records in disparate databases. In this work, we undertake the critical task of employing natural language processing (NLP) techniques and supervised machine learning models to classify textual data in analyst and investigator notes and reports for radicalization behavioral indicators. This effort to generate structured knowledge will build towards an operational capability to assist analysts in rapidly mining law enforcement and intelligence databases for cues and risk indicators. In the near-term, this effort also enables more rapid coding of biographical radicalization profiles to augment a research database of violent extremists and their exhibited behavioral indicators.

Personalized IoT adapt their behavior based on contextual information, such as user behavior and location. Unfortunately, the fact that personalized IoT adapt to user context opens a side-channel that leaks private information about the user. To that end, we start by studying the extent to which a malicious eavesdropper can monitor the actions taken by an IoT system and extract user's private information. In particular, we show two concrete instantiations (in the context of mobile phones and smart homes) of a new category of spyware which we refer to as Context-Aware Adaptation Based Spyware (SpyCon). Experimental evaluations show that the developed SpyCon can predict users' daily behavior with an accuracy of 90.3%. Being a new spyware with no known prior signature or behavior, traditional spyware detection that is based on code signature or system behavior are not adequate to detect SpyCon. We discuss possible detection and mitigation mechanisms that can hinder the effect of SpyCon.

Industry 4.0 and the Internet of Production can increase efficiency and effectiveness of workflows in manufacturing companies and production networks. Despite ubiquitous automation, people are essential in socio-technical cyber-physical production systems due to unique cognitive capabilities, as final arbitrators, or for ethical and legal reasons. However, the design of interfaces between the human-in-the-loop and production systems poses challenges not yet been sufficiently elaborated in research and practice. We present two behavioural studies in the context of inter-company collaboration that show why considering the human-in-the-loop is crucial: The first study shows that information complexity and individual differences shape the overall decision quality. With increasing information complexity, the decision speed decreases and the decision accuracy descends. Consequently, a fine balance between necessary, abundant, and superfluous information must be found. The second experiment studies human decision making in complex environments using a business simulation. We found that correct decision aids can augment the human-in-the-loop's decision making and that these can increase usability, trust, and proft. Yet, incorrect decision support has the opposite effect. Guidelines for designing socio-technical cyber-physical production systems and a research agenda conclude this article.

X-ray baggage security screening is a demanding task for aviation and rail transit security; automatic prohibited item detection in X-ray baggage images can help reduce the work of inspectors. However, as many items are placed too close to each other in the baggages, it is difficult to fully trust the detection results of intelligent prohibited item detection algorithms. In this paper, a human-in-the-loop baggage inspection framework is proposed. The proposed framework utilizes the deep-learning-based algorithm for prohibited item detection to find suspicious items in X-ray baggage images, and select manual examination when the detection algorithm cannot determine whether the baggage is dangerous or safe. The advantages of proposed inspection process include: online to capture new sample images for training incrementally prohibited item detection model, and augmented prohibited item detection intelligence with human-computer collaboration. The preliminary experimental results show, human-in-the-loop process by combining cognitive capabilities of human inspector with the intelligent algorithms capabilities, can greatly improve the efficiency of in-baggage security screening.

To meet the application need of dynamic visualization VR display of 3D time-varying field, this paper designed an immersive visualization VR system of 3D time-varying field based on the Unity 3D framework. To reduce visual confusion caused by 3D time-varying field flow line drawing and improve the quality and efficiency of visualization rendering drawing, deep learning was used to extract features from the mesoscale vortex of the 3D time-varying field. Moreover, the 3D flow line dynamic visualization drawing was implemented through the Unity Visual Effect Graph particle system.