Papallas, Rafael, Dogar, Mehmet R..
2020.
Non-Prehensile Manipulation in Clutter with Human-In-The-Loop. 2020 IEEE International Conference on Robotics and Automation (ICRA). :6723–6729.
We propose a human-operator guided planning approach to pushing-based manipulation in clutter. Most recent approaches to manipulation in clutter employs randomized planning. The problem, however, remains a challenging one where the planning times are still in the order of tens of seconds or minutes, and the success rates are low for difficult instances of the problem. We build on these control-based randomized planning approaches, but we investigate using them in conjunction with human-operator input. In our framework, the human operator supplies a high-level plan, in the form of an ordered sequence of objects and their approximate goal positions. We present experiments in simulation and on a real robotic setup, where we compare the success rate and planning times of our human-in-the-loop approach with fully autonomous sampling-based planners. We show that with a minimal amount of human input, the low-level planner can solve the problem faster and with higher success rates.
Madono, Koki, Nakano, Teppei, Kobayashi, Tetsunori, Ogawa, Tetsuji.
2020.
Efficient Human-In-The-Loop Object Detection using Bi-Directional Deep SORT and Annotation-Free Segment Identification. 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). :1226–1233.
The present study proposes a method for detecting objects with a high recall rate for human-supported video annotation. In recent years, automatic annotation techniques such as object detection and tracking have become more powerful; however, detection and tracking of occluded objects, small objects, and blurred objects are still difficult. In order to annotate such objects, manual annotation is inevitably required. For this reason, we envision a human-supported video annotation framework in which over-detected objects (i.e., false positives) are allowed to minimize oversight (i.e., false negatives) in automatic annotation and then the over-detected objects are removed manually. This study attempts to achieve human-in-the-loop object detection with an emphasis on suppressing the oversight for the former stage of processing in the aforementioned annotation framework: bi-directional deep SORT is proposed to reliably capture missed objects and annotation-free segment identification (AFSID) is proposed to identify video frames in which manual annotation is not required. These methods are reinforced each other, yielding an increase in the detection rate while reducing the burden of human intervention. Experimental comparisons using a pedestrian video dataset demonstrated that bi-directional deep SORT with AFSID was successful in capturing object candidates with a higher recall rate over the existing deep SORT while reducing the cost of manpower compared to manual annotation at regular intervals.
Jobst, Matthias, Liu, Chen, Partzsch, Johannes, Yan, Yexin, Kappel, David, Gonzalez, Hector A., Ji, Yue, Vogginger, Bernhard, Mayr, Christian.
2020.
Event-based Neural Network for ECG Classification with Delta Encoding and Early Stopping. 2020 6th International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP). :1–4.
We present a scalable architecture based on a trained filter bank for input pre-processing and a recurrent neural network (RNN) for the detection of atrial fibrillation in electrocardiogram (ECG) signals, with the focus on enabling a very efficient hardware implementation as application-specific integrated circuit (ASIC). Our already very efficient base architecture is further improved by replacing the RNN with a delta-encoded gated recurrent unit (GRU) and adding a confidence measure (CM) for terminating the computation as early as possible. With these optimizations, we demonstrate a reduction of the processing load of 58 % on an internal dataset while still achieving near state-of-the-art classification results on the Physionet ECG dataset with only 1202 parameters.
Nguyen, Vu, Cabrera, Juan A., Pandi, Sreekrishna, Nguyen, Giang T., Fitzek, Frank H. P..
2020.
Exploring the Benefits of Memory-Limited Fulcrum Recoding for Heterogeneous Nodes. GLOBECOM 2020 - 2020 IEEE Global Communications Conference. :1–6.
Fulcrum decoders can trade off between computational complexity and the number of received packets. This allows heterogeneous nodes to decode at different level of complexity in accordance with their computing power. Variations of Fulcrum codes, like dynamic sparsity and expansion packets (DSEP) have significantly reduced the encoders and decoders' complexity by using dynamic sparsity and expansion packets. However, limited effort had been done for recoders of Fulcrum codes and their variations, limiting their full potential when being deployed at multi-hop networks. In this paper, we investigate the drawback of the conventional Fulcrum recoding and introduce a novel recoding scheme for the family of Fulcrum codes by limiting the buffer size, and thus memory needs. Our evaluations indicate that DSEP recoding mechamism increases the recoding goodput by 50%, and reduces the decoding overhead by 60%-90% while maintaining high decoding goodput at receivers and small memory usage at recoders compared with the conventional Fulcrum recoding. This further reduces the resources needed for Fulcrum codes at the recoders.
Corraro, Gianluca, Bove, Ezio, Canzolino, Pasquale, Cicala, Marco, Ciniglio, Umberto, Corraro, Federico, Di Capua, Gianluigi, Filippone, Edoardo, Garbarino, Luca, Genito, Nicola et al..
2020.
Real-Time HW and Human-in-the-Loop Simulations for the Validation of Detect and Avoid Advanced Functionalities in ATM Future Scenarios. 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC). :1–10.
The full integration of Remotely Piloted Aircraft Systems (RPAS) in non-segregated airspace is one of the major objectives for the worldwide aviation organizations and authorities. However, there are several technological and regulatory issues due to the increase of the air traffic in the next years and to the need of keeping high safety levels. In this framework, a real-time validation environment capable to simulate complex scenarios related to future air traffic management (ATM) conditions is of paramount importance. These facilities allow detailed testing and tuning of new technologies and procedures before executing flight tests. With such motivations, the Italian Aerospace Research Centre has developed the Integrated Simulation Facility (ISF) able to accurately reproduce ATM complex scenarios in real-time with hardware and human in-the-loop simulations, aiming to validate new ATM procedures and innovative system prototypes for RPAS and General Aviation aircraft. In the present work, the ISF facility has been used for reproducing relevant ATM scenarios to validate the functionalities of a Detect and Avoid system (DAA). The results of the ISF test campaign demonstrate the effectiveness of the developed algorithm in the autonomous resolution of mid-air collisions in presence of both air traffic and fixed obstacles (i.e. bad weather areas, no-fly-zone and terrain) and during critical flight phases, thus exceeding the current DAA state-of-the-art.
Matsushita, Haruka, Sato, Kaito, Sakura, Mamoru, Sawada, Kenji, Shin, Seiichi, Inoue, Masaki.
2020.
Rear-wheel steering control reflecting driver personality via Human-In-The-Loop System. 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). :356–362.
One of the typical autonomous driving systems is a human-machine cooperative system that intervenes in the driver operation. The autonomous driving needs to make consideration of the driver individuality in addition to safety. This paper considers a human-machine cooperative system balancing safety with the driver individuality using the Human-In-The-Loop System (HITLS) for rear-wheel steering control. This paper assumes that it is safe for HITLS to follow the target side-slip angle and target angular velocity without conflicts between the controller and driver operations. We propose HITLS using the primal-dual algorithm and the internal model control (IMC) type I-PD controller. In HITLS, the signal expander delimits the human-selectable operating range and the controller cooperates stably the human operation and automated control in that range. The primal-dual algorithm realizes the driver and the signal expander. Our outcomes are the making of the rear-wheel steering system which converges to the target value while reflecting the driver individuality.
Uchida, Hikaru, Matsubara, Masaki, Wakabayashi, Kei, Morishima, Atsuyuki.
2020.
Human-in-the-loop Approach towards Dual Process AI Decisions. 2020 IEEE International Conference on Big Data (Big Data). :3096–3098.
How to develop AI systems that can explain how they made decisions is one of the important and hot topics today. Inspired by the dual-process theory in psychology, this paper proposes a human-in-the-loop approach to develop System-2 AI that makes an inference logically and outputs interpretable explanation. Our proposed method first asks crowd workers to raise understandable features of objects of multiple classes and collect training data from the Internet to generate classifiers for the features. Logical decision rules with the set of generated classifiers can explain why each object is of a particular class. In our preliminary experiment, we applied our method to an image classification of Asian national flags and examined the effectiveness and issues of our method. In our future studies, we plan to combine the System-2 AI with System-1 AI (e.g., neural networks) to efficiently output decisions.
Yeruva, Vijaya Kumari, Chandrashekar, Mayanka, Lee, Yugyung, Rydberg-Cox, Jeff, Blanton, Virginia, Oyler, Nathan A.
2020.
Interpretation of Sentiment Analysis with Human-in-the-Loop. 2020 IEEE International Conference on Big Data (Big Data). :3099–3108.
Human-in-the-Loop has been receiving special attention from the data science and machine learning community. It is essential to realize the advantages of human feedback and the pressing need for manual annotation to improve machine learning performance. Recent advancements in natural language processing (NLP) and machine learning have created unique challenges and opportunities for digital humanities research. In particular, there are ample opportunities for NLP and machine learning researchers to analyze data from literary texts and use these complex source texts to broaden our understanding of human sentiment using the human-in-the-loop approach. This paper presents our understanding of how human annotators differ from machine annotators in sentiment analysis tasks and how these differences can contribute to designing systems for the "human in the loop" sentiment analysis in complex, unstructured texts. We further explore the challenges and benefits of the human-machine collaboration for sentiment analysis using a case study in Greek tragedy and address some open questions about collaborative annotation for sentiments in literary texts. We focus primarily on (i) an analysis of the challenges in sentiment analysis tasks for humans and machines, and (ii) whether consistent annotation results are generated from multiple human annotators and multiple machine annotators. For human annotators, we have used a survey-based approach with about 60 college students. We have selected six popular sentiment analysis tools for machine annotators, including VADER, CoreNLP's sentiment annotator, TextBlob, LIME, Glove+LSTM, and RoBERTa. We have conducted a qualitative and quantitative evaluation with the human-in-the-loop approach and confirmed our observations on sentiment tasks using the Greek tragedy case study.
Zhang, Xinyuan, Liu, Hongzhi, Wu, Zhonghai.
2020.
Noise Reduction Framework for Distantly Supervised Relation Extraction with Human in the Loop. 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC). :1–4.
Distant supervision is a widely used data labeling method for relation extraction. While aligning knowledge base with the corpus, distant supervision leads to a mass of wrong labels which are defined as noise. The pattern-based denoising model has achieved great progress in selecting trustable sentences (instances). However, the writing of relation-specific patterns heavily relies on expert’s knowledge and is a high labor intensity work. To solve these problems, we propose a noise reduction framework, NOIR, to iteratively select trustable sentences with a little help of a human. Under the guidance of experts, the iterative process can avoid semantic drift. Besides, NOIR can help experts discover relation-specific tokens that are hard to think of. Experimental results on three real-world datasets show the effectiveness of the proposed method compared with state-of-the-art methods.
Antunes, Rui Azevedo, Brito Palma, Luís.
2020.
Fitts’ Evaluation of a Developed Human-in-the-Loop Assistive Device. 2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA). :1–6.
In this work, a new human-computer assistive technology gadget designed for people with impairments is evaluated. The developed human-in-the-loop interface device has an embedded assistance controller and can replace the traditional mouse, gamepad and keyboard, enabling human-computer hands-free full access. This work is concerned with the assistive device performance characterization aspects. Based on the experiments carried out, the human-computer performance improvement with the embedded controller is analysed in detail. Results show that adding the human-in-the-loop assistance controller improves human-computer hands-free skills, which is an innovative contribution for the replacement of computer interfaces that depend on the human hands.
Silva, J. Sá, Saldanha, Ruben, Pereira, Vasco, Raposo, Duarte, Boavida, Fernando, Rodrigues, André, Abreu, Madalena.
2019.
WeDoCare: A System for Vulnerable Social Groups. 2019 International Conference on Computational Science and Computational Intelligence (CSCI). :1053–1059.
One of the biggest problems in the current society is people's safety. Safety measures and mechanisms are especially important in the case of vulnerable social groups, such as migrants, homeless, and victims of domestic and/or sexual violence. In order to cope with this problem, we witness an increasing number of personal alarm systems in the market, most of them based on panic buttons. Nevertheless, none of them has got widespread acceptance mainly because of limited Human-Computer Interaction. In the context of this work, we developed an innovative mobile application that recognizes an attack through speech and gesture recognition. This paper describes such a system and presents its features, some of them based on the emerging concept of Human-in-the-Loop Cyber-physical Systems and new concepts of Human-Computer Interaction.
Shin, Ho-Chul.
2019.
Abnormal Detection based on User Feedback for Abstracted Pedestrian Video. 2019 International Conference on Information and Communication Technology Convergence (ICTC). :1036–1038.
In this study, we present the abstracted pedestrian behavior representation and abnormal detection method based on user feedback for pedestrian video surveillance system. Video surveillance data is large in size and difficult to process in real time. To solve this problem, we suggested a method of expressing the pedestrian behavior with abbreviated map. In the video surveillance system, false detection of an abnormal situation becomes a big problem. If surveillance user can guide the false detection case as human in the loop, the surveillance system can learn the case and reduce the false detection error in the future. We suggested user feedback based abnormal pedestrian detection method. By the suggested user feedback algorithm, the false detection can be reduced to less than 0.5%.
Feng, Ri-Chen, Lin, Daw-Tung, Chen, Ken-Min, Lin, Yi-Yao, Liu, Chin-De.
2019.
Improving Deep Learning by Incorporating Semi-automatic Moving Object Annotation and Filtering for Vision-based Vehicle Detection. 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). :2484–2489.
Deep learning has undergone tremendous advancements in computer vision studies. The training of deep learning neural networks depends on a considerable amount of ground truth datasets. However, labeling ground truth data is a labor-intensive task, particularly for large-volume video analytics applications such as video surveillance and vehicles detection for autonomous driving. This paper presents a rapid and accurate method for associative searching in big image data obtained from security monitoring systems. We developed a semi-automatic moving object annotation method for improving deep learning models. The proposed method comprises three stages, namely automatic foreground object extraction, object annotation in subsequent video frames, and dataset construction using human-in-the-loop quick selection. Furthermore, the proposed method expedites dataset collection and ground truth annotation processes. In contrast to data augmentation and data generative models, the proposed method produces a large amount of real data, which may facilitate training results and avoid adverse effects engendered by artifactual data. We applied the constructed annotation dataset to train a deep learning you-only-look-once (YOLO) model to perform vehicle detection on street intersection surveillance videos. Experimental results demonstrated that the accurate detection performance was improved from a mean average precision (mAP) of 83.99 to 88.03.
Boddy, Aaron, Hurst, William, Mackay, Michael, El Rhalibi, Abdennour.
2019.
A Hybrid Density-Based Outlier Detection Model for Privacy in Electronic Patient Record system. 2019 5th International Conference on Information Management (ICIM). :92–96.
This research concerns the detection of unauthorised access within hospital networks through the real-time analysis of audit logs. Privacy is a primary concern amongst patients due to the rising adoption of Electronic Patient Record (EPR) systems. There is growing evidence to suggest that patients may withhold information from healthcare providers due to lack of Trust in the security of EPRs. Yet, patient record data must be available to healthcare providers at the point of care. Ensuring privacy and confidentiality of that data is challenging. Roles within healthcare organisations are dynamic and relying on access control is not sufficient. Through proactive monitoring of audit logs, unauthorised accesses can be detected and presented to an analyst for review. Advanced data analytics and visualisation techniques can be used to aid the analysis of big data within EPR audit logs to identify and highlight pertinent data points. Employing a human-in-the-loop model ensures that suspicious activity is appropriately investigated and the data analytics is continuously improving. This paper presents a system that employs a Human-in-the-Loop Machine Learning (HILML) algorithm, in addition to a density-based local outlier detection model. The system is able to detect 145 anomalous behaviours in an unlabelled dataset of 1,007,727 audit logs. This equates to 0.014% of the EPR accesses being labelled as anomalous in a specialist Liverpool (UK) hospital.
Hung, Benjamin W.K., Muramudalige, Shashika R., Jayasumana, Anura P., Klausen, Jytte, Libretti, Rosanne, Moloney, Evan, Renugopalakrishnan, Priyanka.
2019.
Recognizing Radicalization Indicators in Text Documents Using Human-in-the-Loop Information Extraction and NLP Techniques. 2019 IEEE International Symposium on Technologies for Homeland Security (HST). :1–7.
Among the operational shortfalls that hinder law enforcement from achieving greater success in preventing terrorist attacks is the difficulty in dynamically assessing individualized violent extremism risk at scale given the enormous amount of primarily text-based records in disparate databases. In this work, we undertake the critical task of employing natural language processing (NLP) techniques and supervised machine learning models to classify textual data in analyst and investigator notes and reports for radicalization behavioral indicators. This effort to generate structured knowledge will build towards an operational capability to assist analysts in rapidly mining law enforcement and intelligence databases for cues and risk indicators. In the near-term, this effort also enables more rapid coding of biographical radicalization profiles to augment a research database of violent extremists and their exhibited behavioral indicators.