Biblio

Recent technological advances in developing intelligent telecommunication networks, ultra-compact bendable wireless transceiver chips, adaptive wearable sensors and actuators, and secure computing infrastructures along with the progress made in psychology and neuroscience for understanding neu-rocognitive and computational principles of human behavior combined have paved the way for a new field of research: Tactile Internet with Human-in-the-Loop (TaHiL). This emerging field of transdisciplinary research aims to promote next generation digitalized human-machine interactions in perceived real time. To achieve this goal, mechanisms and principles of human goal-directed multisensory perception and action need to be integrated into technological designs for breakthrough innovations in mobile telecommunication, electronics and materials engineering, as well as computing. This overview highlights key challenges and the frontiers of research in the new field of TaHiL. Revolutionizing the current Internet as a digital infrastructure for sharing visual and auditory information globally, the TaHiL research will enable humans to share tactile and haptic information and thus veridically immerse themselves into virtual, remote, or inaccessible real environments to exchange skills and expertise with other humans or machines for applications in medicine, industry, and the Internet of Skills.

The need for cyber resilience is increasingly important in our technology-dependent society, where computing systems, devices and data will continue to be the target of cyber attackers. Hence, we propose a conceptual framework called ‘Human-in-the-Loop Explainable-AI-Enabled Vulnerability Detection, Investigation, and Mitigation’ (HXAI-VDIM). Specifically, instead of resolving complex scenario of security vulnerabilities as an output of an AI/ML model, we integrate the security analyst or forensic investigator into the man-machine loop and leverage explainable AI (XAI) to combine both AI and Intelligence Assistant (IA) to amplify human intelligence in both proactive and reactive processes. Our goal is that HXAI-VDIM integrates human and machine in an interactive and iterative loop with security visualization that utilizes human intelligence to guide the XAI-enabled system and generate refined solutions.

Human-in-the-Loop has been receiving special attention from the data science and machine learning community. It is essential to realize the advantages of human feedback and the pressing need for manual annotation to improve machine learning performance. Recent advancements in natural language processing (NLP) and machine learning have created unique challenges and opportunities for digital humanities research. In particular, there are ample opportunities for NLP and machine learning researchers to analyze data from literary texts and use these complex source texts to broaden our understanding of human sentiment using the human-in-the-loop approach. This paper presents our understanding of how human annotators differ from machine annotators in sentiment analysis tasks and how these differences can contribute to designing systems for the "human in the loop" sentiment analysis in complex, unstructured texts. We further explore the challenges and benefits of the human-machine collaboration for sentiment analysis using a case study in Greek tragedy and address some open questions about collaborative annotation for sentiments in literary texts. We focus primarily on (i) an analysis of the challenges in sentiment analysis tasks for humans and machines, and (ii) whether consistent annotation results are generated from multiple human annotators and multiple machine annotators. For human annotators, we have used a survey-based approach with about 60 college students. We have selected six popular sentiment analysis tools for machine annotators, including VADER, CoreNLP's sentiment annotator, TextBlob, LIME, Glove+LSTM, and RoBERTa. We have conducted a qualitative and quantitative evaluation with the human-in-the-loop approach and confirmed our observations on sentiment tasks using the Greek tragedy case study.

One of the typical autonomous driving systems is a human-machine cooperative system that intervenes in the driver operation. The autonomous driving needs to make consideration of the driver individuality in addition to safety. This paper considers a human-machine cooperative system balancing safety with the driver individuality using the Human-In-The-Loop System (HITLS) for rear-wheel steering control. This paper assumes that it is safe for HITLS to follow the target side-slip angle and target angular velocity without conflicts between the controller and driver operations. We propose HITLS using the primal-dual algorithm and the internal model control (IMC) type I-PD controller. In HITLS, the signal expander delimits the human-selectable operating range and the controller cooperates stably the human operation and automated control in that range. The primal-dual algorithm realizes the driver and the signal expander. Our outcomes are the making of the rear-wheel steering system which converges to the target value while reflecting the driver individuality.

Personalized IoT adapt their behavior based on contextual information, such as user behavior and location. Unfortunately, the fact that personalized IoT adapt to user context opens a side-channel that leaks private information about the user. To that end, we start by studying the extent to which a malicious eavesdropper can monitor the actions taken by an IoT system and extract user's private information. In particular, we show two concrete instantiations (in the context of mobile phones and smart homes) of a new category of spyware which we refer to as Context-Aware Adaptation Based Spyware (SpyCon). Experimental evaluations show that the developed SpyCon can predict users' daily behavior with an accuracy of 90.3%. Being a new spyware with no known prior signature or behavior, traditional spyware detection that is based on code signature or system behavior are not adequate to detect SpyCon. We discuss possible detection and mitigation mechanisms that can hinder the effect of SpyCon.

Industry 4.0 and the Internet of Production can increase efficiency and effectiveness of workflows in manufacturing companies and production networks. Despite ubiquitous automation, people are essential in socio-technical cyber-physical production systems due to unique cognitive capabilities, as final arbitrators, or for ethical and legal reasons. However, the design of interfaces between the human-in-the-loop and production systems poses challenges not yet been sufficiently elaborated in research and practice. We present two behavioural studies in the context of inter-company collaboration that show why considering the human-in-the-loop is crucial: The first study shows that information complexity and individual differences shape the overall decision quality. With increasing information complexity, the decision speed decreases and the decision accuracy descends. Consequently, a fine balance between necessary, abundant, and superfluous information must be found. The second experiment studies human decision making in complex environments using a business simulation. We found that correct decision aids can augment the human-in-the-loop's decision making and that these can increase usability, trust, and proft. Yet, incorrect decision support has the opposite effect. Guidelines for designing socio-technical cyber-physical production systems and a research agenda conclude this article.

Among the operational shortfalls that hinder law enforcement from achieving greater success in preventing terrorist attacks is the difficulty in dynamically assessing individualized violent extremism risk at scale given the enormous amount of primarily text-based records in disparate databases. In this work, we undertake the critical task of employing natural language processing (NLP) techniques and supervised machine learning models to classify textual data in analyst and investigator notes and reports for radicalization behavioral indicators. This effort to generate structured knowledge will build towards an operational capability to assist analysts in rapidly mining law enforcement and intelligence databases for cues and risk indicators. In the near-term, this effort also enables more rapid coding of biographical radicalization profiles to augment a research database of violent extremists and their exhibited behavioral indicators.

Software systems are increasingly called upon to autonomously manage their goals in changing contexts and environments, and under evolving requirements. In some circumstances, autonomous systems cannot be fully-automated but instead cooperate with human operators to maintain and adapt themselves. Furthermore, there are times when a choice should be made between doing a manual or automated repair. Involving operators in self-adaptation should itself be adaptive, and consider aspects such as the training, attention, and ability of operators. Not only do these aspects change from person to person, but they may change with the same person. These aspects make the choice of whether to involve humans non-obvious. Self-adaptive systems should trade-off whether to involve operators, taking these aspects into consideration along with other business qualities it is attempting to achieve. In this chapter, we identify the various roles that operators can perform in cooperating with self-adapting systems. We focus on humans as effectors-doing tasks which are difficult or infeasible to automate. We describe how we modified our self-adaptive framework, Rainbow, to involve operators in this way, which involved choosing suitable human models and integrating them into the existing utility trade-off decision models of Rainbow. We use probabilistic modeling and quantitative verification to analyze the trade-offs of involving humans in adaptation, and complement our study with experiments to show how different business preferences and modalities of human involvement may result in different outcomes.