Biblio

With the advancement of portable head-mounted displays, interest in educational application of immersive spherical video-based virtual reality (SVVR) has been emerging. However, it remains unclear regarding the effects of immersive SVVR on cognitive and affective outcomes. In this study, we retrieved 58 learning outcomes from 16 studies. A meta-analysis was performed using the random effects model to calculate the effect size. Several important moderators were also examined such as control group treatment, learning outcome type, interaction functionality, content instruction, learning domain, and learner's stage. The results show that immersive SVVR is more effective than other instructional conditions with a medium effect size. The key findings of the moderator analysis are that immersive SVVR has a greater impact on affective outcomes, as well as under the conditions that learning system provides interaction functionality or integrates with content instruction before virtual exploratory learning.

The goal of this study is to find whether exposure to Deepfake videos makes people better at detecting Deepfake videos and whether it is a better strategy against fighting Deepfake. For this study a group of people from Bangladesh has volunteered. This group were exposed to a number of Deepfake videos and asked subsequent questions to verify improvement on their level of awareness and detection in context of Deepfake videos. This study has been performed in two phases, where second phase was performed to validate any generalization. The fake videos are tailored for the specific audience and where suited, are created from scratch. Finally, the results are analyzed, and the study’s goals are inferred from the obtained data.

Deepfake detection techniques have been widely studied to resolve security issues. However, existing techniques mainly focused on RGB channel-based analysis, which still shows incomplete detection accuracy. In this paper, we validate the performance of Gray channel-based deepfake detection. To compare RGB channel-based analysis and Gray channel-based analysis in deepfake detection, we quantitatively measured the performance by using popular CNN models, deepfake datasets, and evaluation indicators. Our experimental results confirm that Gray channel-based deepfake detection outperforms RGB channel-based deepfake detection in terms of accuracy and analysis time.

Recent developments in deepfake technology are increasing new security threats. To solve these issues, various detection methods have been proposed including the methods utilizing biological signals captured by R-PPG. However, existing methods have limitations in terms of detection accuracy and generalized performance. In this paper, we present our approach for R-PPG-based BPM (Beats Per Minute) analysis for effective deepfake detection. With the selected deepfake datasets, we performed (a) comparison and analysis of conditions for BPM processing, and (b) BPM extraction by dividing the face into 16 regions and comparison of BPM in each region. The results showed that our proposed BPM-related properties are effective in deepfake detection.

Face capturing is a task to capture and store the "best" face of each person passing by the monitor. To some extent, it is similar to face tracking, but uses a different criterion and requires a valuable (i.e., high-quality and recognizable) face selection procedure. Face capturing systems play a critical role in public security. When deployed on edge devices, it is capable of reducing redundant storage in data center and speeding up retrieval of a certain person. However, high computation complexity and high repetition rate caused by ID switch errors are major challenges. In this paper, we propose a novel solution to constructing a real-time low-repetition face capturing system on chips. First, we propose a two-stage association algorithm for memory-efficient and accurate face tracking. Second, we propose a fast and reliable face quality estimation algorithm for valuable face selection. Our pipeline runs at over 20fps on Hisiv 3559A SoC with a single NNIE device for neural network inference, while achieving over 95% recall and less than 0.4 repetition rate in real world surveillance videos.

Surveillance systems are widely used in airports, streets, banks, military areas, borders, hospitals, and schools. There are two types of surveillance systems which are real-time systems and offline surveillance systems. Usually, security people track videos on time in monitoring rooms to find out abnormal human activities. Real-time human tracking from videos is very expensive especially in airports, borders, and streets due to the huge number of surveillance cameras. There are a lot of research works have been done for automated surveillance systems. In this paper, we presented a new surveillance system to recognize human activities from several cameras using machine learning algorithms. Sequences of images are collected from cameras using the internet of things technology from indoor or outdoor areas. A feature vector is created for each recognized moving object, then machine learning algorithms are applied to extract moving object activities. The proposed abnormal event detection system gives very promising results which are more than 96% accuracy in Multilayer Perceptron, Iterative Classifier Optimizer, and Random Forest algorithms.

Human action is naturally compositional: humans can easily recognize and perform actions with objects that are different from those used in training demonstrations. In this paper, we study the compositionality of action by looking into the dynamics of subject-object interactions. We propose a novel model which can explicitly reason about the geometric relations between constituent objects and an agent performing an action. To train our model, we collect dense object box annotations on the Something-Something dataset. We propose a novel compositional action recognition task where the training combinations of verbs and nouns do not overlap with the test set. The novel aspects of our model are applicable to activities with prominent object interaction dynamics and to objects which can be tracked using state-of-the-art approaches; for activities without clearly defined spatial object-agent interactions, we rely on baseline scene-level spatio-temporal representations. We show the effectiveness of our approach not only on the proposed compositional action recognition task but also in a few-shot compositional setting which requires the model to generalize across both object appearance and action category.

Phishing sends malicious links or attachments through emails that can perform various functions, including capturing the victim's login credentials or account information. These emails harm the victims, cause money loss, and identity theft. In this paper, we contribute to solving the phishing problem by developing an extension for the Google Chrome web browser. In the development of this feature, we used JavaScript PL. To be able to identify and prevent the fishing attack, a combination of Blacklisting and semantic analysis methods was used. Furthermore, a database for phishing sites is generated, and the text, links, images, and other data on-site are analyzed for pattern recognition. Finally, our proposed solution was tested and compared to existing approaches. The results validate that our proposed method is capable of handling the phishing issue substantially.

Recent advances in generative adversarial networks have made detecting fake videos a challenging task. In this paper, we propose the application of the state-of-the-art attribution based confidence (ABC) metric for detecting deepfake videos. The ABC metric does not require access to the training data or training the calibration model on the validation data. The ABC metric can be used to draw inferences even when only the trained model is available. Here, we utilize the ABC metric to characterize whether a video is original or fake. The deep learning model is trained only on original videos. The ABC metric uses the trained model to generate confidence values. For, original videos, the confidence values are greater than 0.94.

With the rapid progress in recent years, techniques that generate and manipulate multimedia content can now provide a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, and video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. These can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes, fake media created through deep learning tools, and on modern data-driven forensic methods to fight them. The analysis will help highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research.

In this study, it was aimed to recognize the emotional state from facial images using the deep learning method. In the study, which was approved by the ethics committee, a custom data set was created using videos taken from 20 male and 20 female participants while simulating 7 different facial expressions (happy, sad, surprised, angry, disgusted, scared, and neutral). Firstly, obtained videos were divided into image frames, and then face images were segmented using the Haar library from image frames. The size of the custom data set obtained after the image preprocessing is more than 25 thousand images. The proposed convolutional neural network (CNN) architecture which is mimics of LeNet architecture has been trained with this custom dataset. According to the proposed CNN architecture experiment results, the training loss was found as 0.0115, the training accuracy was found as 99.62%, the validation loss was 0.0109, and the validation accuracy was 99.71%.

Augmented reality (AR) has been identified as a key technology to enhance worker utility in the context of increasing automation of repeatable procedures. AR can achieve this by assisting the user in performing complex and frequently changing procedures. Crucial to the success of procedure assistance AR applications is user acceptability, which can be measured by user quality of experience (QoE). An active research topic in QoE is the identification of implicit metrics that can be used to continuously infer user QoE during a multimedia experience. A user's QoE is linked to their affective state. Affective state is reflected in facial expressions. Emotions shown in micro facial expressions resemble those expressed in normal expressions but are distinguished from them by their brief duration. The novelty of this work lies in the evaluation of micro facial expressions as a continuous QoE metric by means of correlation analysis to the more traditional and accepted post-experience self-reporting. In this work, an optimal Rubik's Cube solver AR application was used as a proof of concept for complex procedure assistance. This was compared with a paper-based procedure assistance control. QoE expressed by affect in normal and micro facial expressions was evaluated through correlation analysis with post-experience reports. The results show that the AR application yielded higher task success rates and shorter task durations. Micro facial expressions reflecting disgust correlated moderately to the questionnaire responses for instruction disinterest in the AR application.

Unmanned Aerial Vehicles (UAVs) also known as drones are being used in many applications where they can record or stream videos. One interesting application is the Intelligent Transportation Systems (ITS) and public safety applications where drones record videos and send them to a control center for further analysis. These videos are shared by various clients such as law enforcement or emergency personnel. In such cases, the recording might include faces of civilians or other sensitive information that might pose privacy concerns. While the video can be encrypted and stored in the cloud that way, it can still be accessed once the keys are exposed to third parties which is completely insecure. To prevent such insecurity, in this paper, we propose proxy re-encryption based sharing scheme to enable third parties to access only limited videos without having the original encryption key. The costly pairing operations in proxy re-encryption are not used to allow rapid access and delivery of the surveillance videos to third parties. The key management is handled by a trusted control center, which acts as the proxy to re-encrypt the data. We implemented and tested the approach in a realistic simulation environment using different resolutions under ns-3. The implementation results and comparisons indicate that there is an acceptable overhead while it can still preserve the privacy of drivers and passengers.

Advancements in the AI field unfold tremendous opportunities for society. Simultaneously, it becomes increasingly important to address emerging ramifications. Thereby, the focus is often set on ethical and safe design forestalling unintentional failures. However, cybersecurity-oriented approaches to AI safety additionally consider instantiations of intentional malice – including unethical malevolent AI design. Recently, an analogous emphasis on malicious actors has been expressed regarding security and safety for virtual reality (VR). In this vein, while the intersection of AI and VR (AIVR) offers a wide array of beneficial cross-fertilization possibilities, it is responsible to anticipate future malicious AIVR design from the onset on given the potential socio-psycho-technological impacts. For a simplified illustration, this paper analyzes the conceivable use case of Generative AI (here deepfake techniques) utilized for disinformation in immersive journalism. In our view, defenses against such future AIVR safety risks related to falsehood in immersive settings should be transdisciplinarily conceived from an immersive co-creation stance. As a first step, we motivate a cybersecurity-oriented procedure to generate defenses via immersive design fictions. Overall, there may be no panacea but updatable transdisciplinary tools including AIVR itself could be used to incrementally defend against malicious actors in AIVR.

In this paper, we propose a new method to expose AI-generated fake face images or videos (commonly known as the Deep Fakes). Our method is based on the observations that Deep Fakes are created by splicing synthesized face region into the original image, and in doing so, introducing errors that can be revealed when 3D head poses are estimated from the face images. We perform experiments to demonstrate this phenomenon and further develop a classification method based on this cue. Using features based on this cue, an SVM classifier is evaluated using a set of real face images and Deep Fakes.

High quality face editing in videos is a growing concern and spreads distrust in video content. However, upon closer examination, many face editing algorithms exhibit artifacts that resemble classical computer vision issues that stem from face tracking and editing. As a consequence, we wonder how difficult it is to expose artificial faces from current generators? To this end, we review current facial editing methods and several characteristic artifacts from their processing pipelines. We also show that relatively simple visual artifacts can be already quite effective in exposing such manipulations, including Deepfakes and Face2Face. Since the methods are based on visual features, they are easily explicable also to non-technical experts. The methods are easy to implement and offer capabilities for rapid adjustment to new manipulation types with little data available. Despite their simplicity, the methods are able to achieve AUC values of up to 0.866.

Detecting DeepFake videos are one of the challenges in digital media forensics. This paper proposes a method to detect deepfake videos using Support Vector Machine (SVM) regression. The SVM classifier can be trained with feature points extracted using one of the different feature-point detectors such as HOG, ORB, BRISK, KAZE, SURF, and FAST algorithms. A comprehensive test of the proposed method is conducted using a dataset of original and fake videos from the literature. Different feature point detectors are tested. The result shows that the proposed method of using feature-detector-descriptors for training the SVM can be effectively used to detect false videos.

AI-synthesized face-swapping videos, commonly known as DeepFakes, is an emerging problem threatening the trustworthiness of online information. The need to develop and evaluate DeepFake detection algorithms calls for datasets of DeepFake videos. However, current DeepFake datasets suffer from low visual quality and do not resemble DeepFake videos circulated on the Internet. We present a new large-scale challenging DeepFake video dataset, Celeb-DF, which contains 5,639 high-quality DeepFake videos of celebrities generated using improved synthesis process. We conduct a comprehensive evaluation of DeepFake detection methods and datasets to demonstrate the escalated level of challenges posed by Celeb-DF.

A breakthrough in the emerging use of machine learning and deep learning is the concept of autoencoders and GAN (Generative Adversarial Networks), architectures that can generate believable synthetic content called deepfakes. The threat lies when these low-tech doctored images, videos, and audios blur the line between fake and genuine content and are used as weapons to cause damage to an unprecedented degree. This paper presents a survey of the underlying technology of deepfakes and methods proposed for their detection. Based on a detailed study of all the proposed models of detection, this paper presents SSTNet as the best model to date, that uses spatial, temporal, and steganalysis for detection. The threat posed by document and signature forgery, which is yet to be explored by researchers, has also been highlighted in this paper. This paper concludes with the discussion of research directions in this field and the development of more robust techniques to deal with the increasing threats surrounding deepfake technology.

DeepFake using Generative Adversarial Networks (GANs) tampered videos reveals a new challenge in today's life. With the inception of GANs, generating high-quality fake videos becomes much easier and in a very realistic manner. Therefore, the development of efficient tools that can automatically detect these fake videos is of paramount importance. The proposed DeepFake detection method takes the advantage of the fact that current DeepFake generation algorithms cannot generate face images with varied resolutions, it is only able to generate new faces with a limited size and resolution, a further distortion and blur is needed to match and fit the fake face with the background and surrounding context in the source video. This transformation causes exclusive blur inconsistency between the generated face and its background in the outcome DeepFake videos, in turn, these artifacts can be effectively spotted by examining the edge pixels in the wavelet domain of the faces in each frame compared to the rest of the frame. A blur inconsistency detection scheme relied on the type of edge and the analysis of its sharpness using Haar wavelet transform as shown in this paper, by using this feature, it can determine if the face region in a video has been blurred or not and to what extent it has been blurred. Thus will lead to the detection of DeepFake videos. The effectiveness of the proposed scheme is demonstrated in the experimental results where the “UADFV” dataset has been used for the evaluation, a very successful detection rate with more than 90.5% was gained.

Nowadays, people faced an emerging problem of AI-synthesized face swapping videos, widely known as the DeepFakes. This kind of videos can be created to cause threats to privacy, fraudulence and so on. Sometimes good quality DeepFake videos recognition could be hard to distinguish with people eyes. That's why researchers need to develop algorithms to detect them. In this work, we present overview of indicators that can tell us about the fact that face swapping algorithms were used on photos. Main purpose of this paper is to find algorithm or technology that can decide whether photo was changed with DeepFake technology or not with good accuracy.

Deepfake imagery that contains altered faces has become a threat to online content. Current anti-deepfake approaches usually do so by detecting image anomalies, such as visible artifacts or inconsistencies. However, with deepfake advances, these visual artifacts are becoming harder to detect. In this paper, we show that one can use biometric eyebrow matching as a tool to detect manipulated faces. Our method could provide an 0.88 AUC and 20.7% EER for deepfake detection when applied to the highest quality deepfake dataset, Celeb-DF.

Video streams acquired from thermal cameras are proven to be beneficial in diverse number of fields including military, healthcare, law enforcement, and security. Despite the hype, thermal imaging is increasingly affected by poor resolution, where it has expensive optical sensors and inability to attain optical precision. In recent years, deep learning based super-resolution algorithms are developed to enhance the video frame resolution at high accuracy. This paper presents a comparative analysis of super resolution (SR) techniques based on deep neural networks (DNN) that are applied on thermal video dataset. SRCNN, EDSR, Auto-encoder, and SRGAN are also discussed and investigated. Further the results on benchmark thermal datasets including FLIR, OSU thermal pedestrian database and OSU color thermal database are evaluated and analyzed. Based on the experimental results, it is concluded that, SRGAN has delivered a superior performance on thermal frames when compared to other techniques and improvements, which has the ability to provide state-of-the art performance in real time operations.

Over the years, technology has reformed the perception of the world related to security concerns. To tackle security problems, we proposed a system capable of detecting security alerts. System encompass audio events that occur as an outlier against background of unusual activity. This ambiguous behaviour can be handled by auditory classification. In this paper, we have discussed two techniques of extracting features from sound data including: time-based and signal based features. In first technique, we preserve time-series nature of sound, while in other signal characteristics are focused. Convolution neural network is applied for categorization of sound. Major aim of research is security challenges, so we have generated data related to surveillance in addition to available datasets such as UrbanSound 8k and ESC-50 datasets. We have achieved 94.6% accuracy for proposed methodology based on self-generated dataset. Improved accuracy on locally prepared dataset demonstrates novelty in research.

In this paper, we present an improved approach to transfer style for videos based on semantic segmentation. We segment foreground objects and background, and then apply different styles respectively. A fully convolutional neural network is used to perform semantic segmentation. We increase the reliability of the segmentation, and use the information of segmentation and the relationship between foreground objects and background to improve segmentation iteratively. We also use segmentation to improve optical flow, and apply different motion estimation methods between foreground objects and background. This improves the motion boundaries of optical flow, and solves the problems of incorrect and discontinuous segmentation caused by occlusion and shape deformation.