Biblio

With the increasing number of distributed energy sources and the growing demand for free exchange of energy, Energy internet (EI) is confronted with great challenges of persistent connection, stable transmission, real-time interaction, and security. The new definition of metaverse in the EI field is proposed as a potential solution for these challenges by establishing a massive and comprehensive fusion 3D network, which can be considered as the advanced stage of EI. The main characteristics of the metaverse such as reality to virtualization, interaction, persistence, and immersion are introduced. Specifically, we present the key enabling technologies of the metaverse including virtual reality, artificial intelligence, blockchain, and digital twin. Meanwhile, the potential applications are presented from the perspectives of immersive user experience, virtual power station, management, energy trading, new business, device maintenance. Finally, some challenges of metaverse in EI are concluded.

Metaverse opens up a new social networking paradigm where people can experience a real interactive feeling without physical space constraints. Social interactions are gradually evolving from text combined with pictures and videos to 3-dimensional virtual reality, making the social experience increasingly physical, implying that more metaverse applications with immersive experiences will be developed in the future. However, the increasing data dimensionality and volume for new metaverse applications present a significant challenge in data acquisition, security, and sharing. Furthermore, metaverse applications require high capacity and ultrareliability for the wireless system to guarantee the quality of user experience, which cannot be addressed in the current fifth-generation system. Therefore, reaching the metaverse is dependent on the revolution in the sixth-generation (6G) wireless communication, which is expected to provide low-latency, high-throughput, and secure services. This article provides a comprehensive view of metaverse applications and investigates the fundamental technologies for the 6G toward metaverse.

In recent years, technological advancements in the AI and VR fields have increasingly often been paired with considerations on ethics and safety aimed at mitigating unintentional design failures. However, cybersecurity-oriented AI and VR safety research has emphasized the need to additionally appraise instantiations of intentional malice exhibited by unethical actors at pre- and post-deployment stages. On top of that, in view of ongoing malicious deepfake developments that can represent a threat to the epistemic security of a society, security-aware AI and VR design strategies require an epistemically-sensitive stance. In this vein, this paper provides a theoretical basis for two novel AIVR safety research directions: 1) VR as immersive testbed for a VR-deepfake-aided epistemic security training and 2) AI as catalyst within a deepfake-aided so-called cyborgnetic creativity augmentation facilitating an epistemically-sensitive threat modelling. For illustration, we focus our use case on deepfake text – an underestimated deepfake modality. In the main, the two proposed transdisciplinary lines of research exemplify how AIVR safety to defend against unethical actors could naturally converge toward AIVR ethics whilst counteracting epistemic security threats.

This Innovative Practice Work-in-Progress paper presents a virtual, proactive, and collaborative learning paradigm that can engage learners with different backgrounds and enable effective retention and transfer of the multidisciplinary AI-cybersecurity knowledge. While progress has been made to better understand the trustworthiness and security of artificial intelligence (AI) techniques, little has been done to translate this knowledge to education and training. There is a critical need to foster a qualified cybersecurity workforce that understands the usefulness, limitations, and best practices of AI technologies in the cybersecurity domain. To address this import issue, in our proposed learning paradigm, we leverage multidisciplinary expertise in cybersecurity, AI, and statistics to systematically investigate two cohesive research and education goals. First, we develop an immersive learning environment that motivates the students to explore AI/machine learning (ML) development in the context of real-world cybersecurity scenarios by constructing learning models with tangible objects. Second, we design a proactive education paradigm with the use of hackathon activities based on game-based learning, lifelong learning, and social constructivism. The proposed paradigm will benefit a wide range of learners, especially underrepresented students. It will also help the general public understand the security implications of AI. In this paper, we describe our proposed learning paradigm and present our current progress of this ongoing research work. In the current stage, we focus on the first research and education goal and have been leveraging cost-effective Minecraft platform to develop an immersive learning environment where the learners are able to investigate the insights of the emerging AI/ML concepts by constructing related learning modules via interacting with tangible AI/ML building blocks.

With novel 3D imaging technology based on computed tomography (CT) set to replace the current 2D X-ray systems, airports face the challenge of adequately preparing airport security officers (screeners) through knowledge building. Virtual reality (VR) bears the potential to greatly facilitate this process by allowing learners to experience and engage in immersive virtual scenarios as if they were real. However, while general aspects of immersion have been explored frequently, less is known about the benefits of immersive technology for instructional purposes in practical settings such as airport security.In the present study, we evaluated how different display technologies (2D vs VR) and segmentation (system-paced vs learner-paced) affected screeners' objective and subjective knowledge gain, cognitive load, as well as aspects of motivation and technology acceptance. By employing a 2 x 2 between-subjects design, four experimental groups experienced uniform learning material featuring information about 3D CT technology and its application in airport security: 2D system-paced, 2D learner-paced, VR system-paced, and VR learner-paced. The instructional material was presented as an 11 min multimedia lesson featuring words (i.e., narration, onscreen text) and pictures in dynamic form (i.e., video, animation). Participants of the learner-paced groups were prompted to initialize the next section of the multimedia lesson by pressing a virtual button after short segments of information. Additionally, a control group experiencing no instructional content was included to evaluate the effectiveness of the instructional material. The data was collected at an international airport with screeners having no prior 3D CT experience (n=162).The results show main effects on segmentation for objective learning outcomes (favoring system-paced), germane cognitive load on display technology (supporting 2D). These results contradict the expected benefits of VR and segmentation, respectively. Overall, the present study offers valuable insight on how to implement instructional material for a practical setting.

Intelligent Virtual Agents (IVAs) have become ubiquitous in our daily lives, displaying increased complexity of form and function. Initial IVA development efforts provided basic functionality to suit users' needs, typically in work or educational settings, but are now present in numerous contexts in more realistic, complex forms. In this paper, we focus on personalization of embodied human intelligent virtual agents to assist individuals as part of physical training “exergames”.

In this work, the use of an undercover conversational agent, acting as a participative student in a synchronous virtual reality distance learning scenario is proposed to stimulate social interaction between teacher and students. The outcome of an exploratory user study indicated that the undercover conversational agent is capable of fostering interaction, relieving social pressure, and overall leading to a more satisfactory and engaging learning experience without sacrificing learning performance.

The recent experience in the use of virtual reality (VR) technology has shown that users prefer Electromyography (EMG) sensor-based controllers over hand controllers. The results presented in this paper show the potential of EMG-based controllers, in particular the Myo armband, to identify a computer system user. In the first scenario, we train various classifiers with 25 keyboard typing movements for training and test with 75. The results with a 1-dimensional convolutional neural network indicate that we are able to identify the user with an accuracy of 93% by analyzing only the EMG data from the Myo armband. When we use 75 moves for training, accuracy increases to 96.45% after cross-validation.

As autonomous service robots will become increasingly ubiquitous in our daily lives, human-robot conflicts will become more likely when humans and robots share the same spaces and resources. This thesis investigates the conflict resolution of robots and humans in everyday conflicts in the domestic and public context. Hereby, the acceptability, trustworthiness, and effectiveness of verbal and non-verbal strategies for the robot to solve the conflict in its favor are evaluated. Based on the assumption of the Media Equation and CASA paradigm that people interact with computers as social actors, robot conflict resolution strategies from social psychology and human-machine interaction were derived. The effectiveness, acceptability, and trustworthiness of those strategies were evaluated in online, virtual reality, and laboratory experiments. Future work includes determining the psychological processes of human-robot conflict resolution in further experimental studies.

Conversational agents (CAs) have been widely adopted for various purposes, ranging from personal assistants to health information providers. While the research on CAs is growing rapidly, less attention is paid to CAs in virtual reality (VR) environments with respect to how the design of these agents influences their trustworthiness as perceived by users, which is key to the adoption and use of VR products featuring CAs. Accordingly, this position paper conceptualizes an immersive model of user trust in CAs in VR. The model is centered around users’ sense of co-presence with CAs in VR, which is influenced by the agents’ embodiment, expressiveness, and responsiveness.

In order to improve the accuracy and efficiency of transmission line safety early warning, a transmission line safety early warning system based on big data variable analysis is proposed. Firstly, the overall architecture of the system is designed under the B / S architecture. Secondly, in the hardware part of the system, the security data real-time monitoring module, data transmission module and security warning module are designed to meet the functional requirements of the system. Finally, in the system software design part, the big data variable analysis method is used to calculate the hidden danger of transmission line safety, so as to improve the effectiveness of transmission safety early warning. The experimental results show that, compared with the traditional security early warning system, the early warning accuracy and efficiency of the designed system are significantly improved, which can ensure the safe operation of the transmission line.

To meet the application need of dynamic visualization VR display of 3D time-varying field, this paper designed an immersive visualization VR system of 3D time-varying field based on the Unity 3D framework. To reduce visual confusion caused by 3D time-varying field flow line drawing and improve the quality and efficiency of visualization rendering drawing, deep learning was used to extract features from the mesoscale vortex of the 3D time-varying field. Moreover, the 3D flow line dynamic visualization drawing was implemented through the Unity Visual Effect Graph particle system.

Construction of immersive architectural wisdom guiding environment based on virtual reality is studied in this paper. Emerging development of the computer smart systems have provided the engineers a novel solution for the platform construction. Network virtualization is currently the most unclear and controversial concept in the industry regarding the definition of virtualization subdivisions. To improve the current study, we use the VR system to implement the platform. The wisdom guiding environment is built through the virtual data modelling and the interactive connections. The platform is implemented through the software. The test on the data analysis accuracy and the interface optimization is conducted.

The project uses 3D immersive technology to innovatively apply virtual reality technology to the monitoring field, and proposes the concept and technical route of remote 3D immersive intelligent control. A design scheme of a three-dimensional immersive remote somatosensory intelligent controller is proposed, which is applied to the remote three-dimensional immersive control of a crawler mobile robot, and the test and analysis of the principle prototype are completed.

This work presents a system which allows image data and extracted features from a real-world location to be captured and modelled in a Virtual Reality (VR) environment combined with Augmented Situated Visualizations (ASV) overlaid and registered in a virtual environment. Combining these technologies with techniques from Data Science and Artificial Intelligence (AI)(such as image analysis and 3D reconstruction) allows the creation of a setting where remote locations can be modelled and interacted with from anywhere in the world. This Enhanced Photorealistic Immersive (EPI) system is highly adaptable to a wide range of use cases and users as it can be utilized to model and interact with any environment which can be captured as image data (such as training for operation in hazardous environments, accessibility solutions for exploration of historical/tourism locations and collaborative learning environments). A use case example focused on a structural examination of railway tunnels along with a pilot study is presented, which can demonstrate the usefulness of the EPI system.

With the advancement of portable head-mounted displays, interest in educational application of immersive spherical video-based virtual reality (SVVR) has been emerging. However, it remains unclear regarding the effects of immersive SVVR on cognitive and affective outcomes. In this study, we retrieved 58 learning outcomes from 16 studies. A meta-analysis was performed using the random effects model to calculate the effect size. Several important moderators were also examined such as control group treatment, learning outcome type, interaction functionality, content instruction, learning domain, and learner's stage. The results show that immersive SVVR is more effective than other instructional conditions with a medium effect size. The key findings of the moderator analysis are that immersive SVVR has a greater impact on affective outcomes, as well as under the conditions that learning system provides interaction functionality or integrates with content instruction before virtual exploratory learning.

Stress-related disorders are increasing because of work load, forces in teamwork, surroundings pressures and health related conditions. Thus, to avoid people living under heavy stress and develop more severe stress-related disorders, different internet and applications of stress management interventions are offered. Mobile applications with self-assessed health, burnout-scores and well-being are commonly used as outcome measures. Few studies have used sickleave to compare effects of stress interventions. A new approach is to use nature and garden in a multimodal stress management context. This study aimed to explore the effects of immersive and non-immersive games application by using nature theme virtual stress therapy in reducing stress level. Two weeks’ of experiments had involved 18 participants. Nine (9) of them were invited to join the first experiment which focused on immersive virtual reality (VR) experience. Their Blood Volume Pulse with Heart Rate (BVP+HR) and Skin Conductance (SC) were recorded using BioGraph Infiniti Biofeedback System that comes with three (3) sensors attached to the fingers. The second experiment were joined by another nine (9) participants. This experiment was testing on non-immersive desktop control experience. The same protocol measurements were taken which are BVP+HR and SC. Participants were given the experience to feel and get carried into the virtual nature as a therapy so that they will reduce stress. The result of this study points to whether immersive or non-immersive VR display using nature theme virtual therapy would reduce individuals stress level. After conducted series of experiments, results showed that both immersive and non-immersive VR display reduced stress level. However, participants were satisfied of using the immersive version as it provided a 360 degree of viewing, immersed experiences and feeling engaged. Thus, this showed and proved that applications developed with nature theme affect successfully reduce stress level no matter it is put in immersive or non-immersive display.

Since a training system using VR can reproduce an actual training environment, training systems have been studied in commercial fields such as medical care and construction. This immersive experience in a virtual space can have a great effect on learning a second language. In this paper, we propose an immersive learning system that learns phrases used in the customer service industry in the customer service experience. We asked the subjects to experience the system, measured the effects of learning, and evaluated the system. Evaluating the learning effect of phrases used in customer service English on 8 students, all student achieved good learning results. Besides, to evaluate the usability of the system, the VR system was evaluated by performing SSQ to measure VR sickness shows this system doesn't cause virtual sickness, SUS to measure usability shows this system evaluation is higher than average system, and IPQ to measure presence in an immersive space shows this system gives average virtual reality experience.

Replacing real life with the virtual space has long ceased to be a theory. Among the whole variety of visualization, systems that allow projecting non-existent objects into real-world space are especially distinguished. Thus, augmented reality technology has found its application in many different fields. The article discusses the general concepts and principles of building augmented reality systems.

In this article we propose the architecture of a system in which its central objective is focused on creating a complete framework for creating outdoor environments of Augmented Reality (AR) and Virtual Reality (VR) allowing its users to digitize reality for hypermedia format. Subsequently, there will be an internal process with the objective of merging / grouping these 3D models, thus enabling clear and intuitive navigation within infinite virtual realities (based on the captured real world). In this way, the user is able to create points of interest within their parallel realities, being able to navigate and traverse their new worlds through these points.

This prototype demonstrates the viability of manipulating both physical and virtual objects with the same tool in order to maintain object permanence across both modes of interaction. Using oppositional force feedback, provided by a servo, and an augmented visual interface, provided by the user’s smartphone, this tool simulates the look and feel of a physical object within an augmented environment. Additionally, the tool is also able to manipulate physical objects that are not part of the augmented reality, such as a physical nut. By integrating both modes of interaction into the same tool, users can fluidly move between these different modes of interaction, manipulating both physical and virtual objects as the need arises. By overlaying this kind of visual and haptic augmentation onto a common tool such as a pair of pliers, we hope to further explore scenarios for collaborative telepresence in future work.

The measurement of room acoustics plays a wide role in audio research, from physical acoustics modelling and virtual reality applications to speech enhancement. While vast literature exists on position-dependent room acoustics and coupling of rooms, little has explored the transition from one room to its neighbour. This paper presents the measurement and analysis of a dataset of spatial room impulse responses for the transition between four coupled room pairs. Each transition consists of 101 impulse responses recorded using a fourth-order spherical microphone array in 5 cm intervals, both with and without a continuous line-of-sight between the source and microphone. A numerical analysis of the room transitions is then presented, including direct-to-reverberant ratio and direction of arrival estimations, along with potential applications and uses of the dataset.

Designing an authentication method is a crucial component to secure privacy in information systems. Virtual Reality (VR) is a new interaction platform, in which the users can interact with natural behaviours (e.g. hand, gaze, head, etc.). In this work, we propose a novel authentication method in which user can perform hand motion in an eyes-free manner. We evaluate the usability and security between eyes-engage and eyes-free input with a pilot study. The initial result revealed our purposed method can achieve a trade-off between usability and security, showing a new way to behaviour-based authentication in VR.

Crime scene investigation and preservation are fundamentally the pillars of forensics. Numerous cases have been discussed in this paper where mishandling of evidence or improper investigation leads to lengthy trials and even worse incorrect verdicts. Whether the problem is lack of training of first responders or any other scenario, it is essential for police officers to properly preserve the evidence. Second problem is the criminal profiling where each district department has its own method of storing information about criminals. ARCRIME intends to digitally transform the way police combat crime. It will allow police officers to create a copy of the scene of crime so that it can be presented in courts or in forensics labs. It will be in the form of wearable glasses for officers on site whereas officers during training will be wearing a headset. The trainee officers will be provided with simulations of cases which have already been resolved. Officers on scene would be provided with intelligence about the crime and the suspect they are interviewing. They would be able to create a case file with audio recording and images which can be digitally sent to a prosecution lawyer. This paper also explores the risks involved with ARCRIME and also weighs in their impact and likelihood of happening. Certain contingency plans have been highlighted in the same section as well to respond to emergency situations.

The goal of this project is to create a realistic VR experience of solitary confinement and study its impact on users. Although there have been active debates and studies on this subject, very few people have personal experience of solitary confinement. Our first aim is to create such an experience in VR to raise the awareness of solitary confinement. We also want to conduct user studies to compare the VR solitary confinement experience with other types of media experiences, such as films or personal narrations. Finally, we want to study people’s sense of time in such a VR environment.