Biblio

This paper builds an Augmented Reality Sandbox (AR Sandbox) system based on augmented reality technology, and performs a 3D reconstruction for the sandbox terrain using the depth sensor Microsoft Kinect in the AR Sandbox, as an entry point to pave the way for later development of related metaverse applications, such as the metaverse architecting and visual interactive modeling. The innovation of this paper is that for the AR Sandbox scene, a 3D reconstruction method based on depth sensor is proposed, which can automatically cut off the edge of the sandbox table in Kinect field of view, and accurately and completely reconstruct the sandbox terrain in Matlab.

Steganography is one of the techniques for secure transformation of data which aims at hiding information inside other media in such a way that no one will notice. The cover media that can accommodate secret information include text, audio, image, and video. Images are the most popular covering media in steganography, due to the fact that, they are heavily used in daily applications and have high redundancy in representation. In this paper, we propose an adaptive steganography algorithm for hiding information in RGB images. To minimize visual perceptible distortion, the proposed algorithm uses edge pixels for embedding data. It detects the edge pixels in the image using the Sobel filter. Then, the message is embedded into the LSBs of the blue channel of the edge pixels. To resist statistical attacks, the distribution of the blue channel of the edge pixels is used when embedding data in the cover image. The experimental results showed that the algorithm offers high capacity for hiding data in cover images; it does not distort the quality of the stego image; it is robust enough against statistical attacks; and its execution time is short enough for online data transfer. Also, the results showed that the proposed algorithm outperforms similar approaches in all evaluation metrics.

Python libraries are widely used for machine learning and scientific computing tasks today. APIs in Python libraries are deprecated due to feature enhancements and bug fixes in the same way as in other languages. These deprecated APIs are discouraged from being used in further software development. Manually detecting and replacing deprecated APIs is a tedious and time-consuming task due to the large number of API calls used in the projects. Moreover, the lack of proper documentation for these deprecated APIs makes the task challenging. To address this challenge, we propose an algorithm and a tool APIScanner that automatically detects deprecated APIs in Python libraries. This algorithm parses the source code of the libraries using abstract syntax tree (ASTs) and identifies the deprecated APIs via decorator, hard-coded warning or comments. APIScanner is a Visual Studio Code Extension that highlights and warns the developer on the use of deprecated API elements while writing the source code. The tool can help developers to avoid using deprecated API elements without the execution of code. We tested our algorithm and tool on six popular Python libraries, which detected 838 of 871 deprecated API elements. Demo of APIScanner: https://youtu.be/1hy\_ugf-iek. Documentation, tool, and source code can be found here: https://rishitha957.github.io/APIScanner.

Advances in artificial intelligence, machine learning, and robotics have profoundly impacted the field of autonomous navigation and driving. However, sensor spoofing attacks can compromise critical components and the control mechanisms of mobile robots. Therefore, understanding vulnerabilities in autonomous driving and developing countermeasures remains imperative for the safety of unmanned vehicles. Hence, we demonstrate cross-validation techniques for detecting spoofing attacks on the sensor data in autonomous driving in this work. First, we discuss how visual and inertial odometry (VIO) algorithms can provide a root-of-trust during navigation. Then, we develop examples for sensor data spoofing attacks using the open-source driving dataset. Next, we design an attack detection technique using VIO algorithms that cross-validates the navigation parameters using the IMU and the visual data. Following, we consider hardware-dependent attack survival mechanisms that support an autonomous system during an attack. Finally, we also provide an example of spoofing survival technique using on-board hardware oscillators. Our work demonstrates the applicability of classical mobile robotics algorithms and hardware security primitives in defending autonomous vehicles from targeted cyber attacks.

In order to improve the security and reliability of information system and reduce the risk of vulnerability intrusion and attack, an automatic patch installation method of operating systems based on deep learning is proposed, If the installation is successful, the basic information of the system will be returned to the visualization server. If the installation fails, it is recommended to upgrading manually and display it on the patch detection visualization server. Through the practical application of statistical analysis, the statistical results show that the proposed method is significantly better than the original and traditional installation methods, which can effectively avoid the problem of client repeated download, and greatly improve the success rate of patch automatic upgrades. It effectively saves the upgrade cost and ensures the security and reliability of the information system.

Deep learning (DL) classifiers are often unstable in that they may change significantly when retested on perturbed images or low quality images. This paper adds to the fundamental body of work on the robustness of DL classifiers. We introduce a new two-dimensional benchmarking matrix to evaluate robustness of DL classifiers, and we also innovate a four-quadrant statistical visualization tool, including minimum accuracy, maximum accuracy, mean accuracy, and coefficient of variation, for benchmarking robustness of DL classifiers. To measure robust DL classifiers, we create comprehensive 69 benchmarking image sets, including a clean set, sets with single factor perturbations, and sets with two-factor perturbation conditions. After collecting experimental results, we first report that using two-factor perturbed images improves both robustness and accuracy of DL classifiers. The two-factor perturbation includes (1) two digital perturbations (salt & pepper noise and Gaussian noise) applied in both sequences, and (2) one digital perturbation (salt & pepper noise) and a geometric perturbation (rotation) applied in both sequences. All source codes, related image sets, and results are shared on the GitHub website at https://github.com/caperock/robustai to support future academic research and industry projects.

A malware detection model based on semi-supervised learning is proposed in the paper. Our model includes mainly three parts: malware visualization, feature extraction, and classification. Firstly, the malware visualization converts malware into grayscale images; then the features of the images are extracted to reflect the coding patterns of malware; finally, a collaborative learning model is applied to malware detections using both labeled and unlabeled software samples. The proposed model was evaluated based on two commonly used benchmark datasets. The results demonstrated that compared with traditional methods, our model not only reduced the cost of sample labeling but also improved the detection accuracy through incorporating unlabeled samples into the collaborative learning process, thereby achieved higher classification performance.

This Wiping Tool is an anti-forensic tool that is built to wipe data permanently from laptop's storage. This tool is capable to ensure the data from being recovered with any recovery tools. The objective of building this wiping tool is to maintain the confidentiality and integrity of the data from unauthorized access. People tend to delete the file in normal way, however, the file face the risk of being recovered. Hence, the integrity and confidentiality of the deleted file cannot be protected. Through wiping tools, the files are overwritten with random strings to make the files no longer readable. Thus, the integrity and the confidentiality of the file can be protected. Regarding wiping tools, nowadays, lots of wiping tools face issue such as data breach because the wiping tools are unable to delete the data permanently from the devices. This situation might affect their main function and a threat to their users. Hence, a new wiping tool is developed to overcome the problem. A new wiping tool named Data Wiping tool is applying two wiping techniques. The first technique is Randomized Data while the next one is enhancing wiping technique, known as ByteEditor. ByteEditor is a combination of two different techniques, byte editing and byte deletion. With the implementation of Object-Oriented methodology, this wiping tool is built. This methodology consists of analyzing, designing, implementation and testing. The tool is analyzed and compared with other wiping tools before the designing of the tool start. Once the designing is done, implementation phase take place. The code of the tool is created using Visual Studio 2010 with C\# language and being tested their functionality to ensure the developed tool meet the objectives of the project. This tool is believed able to contribute to the development of wiping tools and able to solve problems related to other wiping tools.

Clustering artworks is difficult for several reasons. On the one hand, recognizing meaningful patterns in accordance with domain knowledge and visual perception is extremely difficult. On the other hand, applying traditional clustering and feature reduction techniques to the highly dimensional pixel space can be ineffective. To address these issues, we propose to use a deep convolutional embedding model for digitized painting clustering, in which the task of mapping the raw input data to an abstract, latent space is jointly optimized with the task of finding a set of cluster centroids in this latent feature space. Quantitative and qualitative experimental results show the effectiveness of the proposed method. The model is also capable of outperforming other state-of-the-art deep clustering approaches to the same problem. The proposed method can be useful for several art-related tasks, in particular visual link retrieval and historical knowledge discovery in painting datasets.

Due to respectively limited training data, different entities addressing the same vision task based on certain sensitive images may not train a robust deep network. This paper introduces a new vision task where various entities share task-specific image data to enlarge each other's training data volume without visually disclosing sensitive contents (e.g. illegal images). Then, we present a new structure-based training regime to enable different entities learn task-specific and reconstruction-proof image representations for image data sharing. Specifically, each entity learns a private Deep Poisoning Module (DPM) and insert it to a pre-trained deep network, which is designed to perform the specific vision task. The DPM deliberately poisons convolutional image features to prevent image reconstructions, while ensuring that the altered image data is functionally equivalent to the non-poisoned data for the specific vision task. Given this equivalence, the poisoned features shared from one entity could be used by another entity for further model refinement. Experimental results on image classification prove the efficacy of the proposed method.

This paper proposes a novel model to detect Deep-Fakes, which are hyper-realistic fake videos generated by advanced AI algorithms involving facial superimposition. With a growing number of DeepFakes involving prominent political figures that hold a lot of social capital, their misuse can lead to drastic repercussions. These videos can not only be used to circulate false information causing harm to reputations of individuals, companies and countries, but also has the potential to cause civil unrest through mass hysteria. Hence it is of utmost importance to detect these DeepFakes and promptly curb their spread. We therefore propose a CNN-based model that learns inherently distinct patterns that change between a DeepFake and a real video. These distinct features include pixel distortion, inconsistencies with facial superimposition, skin colour differences, blurring and other visual artifacts. The proposed model has trained a CNN (Convolutional Neural Network), to effectively distinguish DeepFake videos using a frame-based approach based on aforementioned distinct features. Herein, the proposed work demonstrates the viability of our model in effectively identifying Deepfake faces in a given video source, so as to aid security applications employed by social-media platforms in credibly tackling the ever growing threat of Deepfakes, by effectively gauging the authenticity of videos, so that they may be flagged or ousted before they can cause irreparable harm.

In this paper, we will describe the design and implementation of a secure payment system based on QR codes. These QR codes have been extensively used in recent years since they speed up the payment process and provide users with ultimate convenience. However, as convenient as they may sound, QR-based online payment systems are vulnerable to different types of attacks. Therefore, transaction processing needs to be secure enough to protect the integrity and confidentiality of every payment process. Moreover, the online payment system must provide authenticity for both the sender and receiver of each transaction. In this paper, the security of the proposed QR-based system is provided using visual cryptography. The proposed system consists of a mobile application and a payment gateway server that implements visual cryptography. The application provides a simple and user-friendly interface for users to carry out payment transactions in user-friendly secure environment.

The ever-evolving computers has its implications on the data and information and the threats that they are exposed to. With the exponential growth of internet, the chances of data breach are highly likely as unauthorized and ill minded users find new ways to get access to the data that they can use for their plans. Most of the systems today have well designed measures that examine the information for any abnormal behavior (Zero Day Attacks) compared to what has been seen and experienced over the years. These checks are done based on a predefined identity (signature) of information. This is being termed as Intrusion Detection Systems (IDS). The concept of IDS revolves around validation of data and/or information and detecting unauthorized access attempts with an intention of manipulating data. High Performance Soft Computing (HPSC) aims to internalize cumulative adoption of traditional and modern attempts to breach data security and expose it to high scale damage and altercations. Our effort in this paper is to emphasize on the multifaceted tactic and rationalize important functionalities of IDS available at the disposal of HPSC.

Photo Response Non-Uniformity (PRNU) noise obtained from images or videos is used as a camera fingerprint to attribute visual objects captured by a camera. The PRNU-based source attribution method, however, fails when there is misalignment between the fingerprint and the query object. One example of such a misalignment, which has been overlooked in the field, is caused by the in-camera resizing technique that a video may have been subjected to. This paper investigates the attribution of visual media in the context of matching a video query object to an image fingerprint or vice versa. Specifically this paper focuses on improving camera attribution performance by taking into account the effects of binning, a commonly used in-camera resizing technique applied to video. We experimentally show that the True Positive Rate (TPR) obtained when binning is considered is approximately 3% higher.

Implementing the color images encryption is a challenging field of the research for IOT applications. An exponential growth in imaging cameras in IOT uses makes it critical to design the robust image security algorithms. It is also observed that performance of existing encryption methods degrades under the presence of noisy environments. This is the major concern of evaluating the encryption method in this paper. The prime concern of this paper is to design the fast efficient color images encryption algorithm by designing an efficient and robustness RSA encryption algorithm. Method takes the advantage of both preprocessing and the Gaussian pyramid (GP) approach for encryption. To improve the performance it is proposed to use the LAB color space and implement the RSA encryption on luminance (L) component using the GP domain. The median filter and image sharpening is used for preprocessing. The goal is to improve the performance under highly noisy imaging environment. The performance is compared based on the crypto weights and on the basis of visual artifacts and entropy analysis. The decrypted outputs are again converted to color image output. Using the LAB color space is expected to improve the entropy performance of the image. Result of proposed encryption method is evaluated under the different types of the noisy attacks over the color images and also performance is compared with state of art encryption methods. Significant improvement speed of the algorithm is compared in terms of the elapsed time

Replacing real life with the virtual space has long ceased to be a theory. Among the whole variety of visualization, systems that allow projecting non-existent objects into real-world space are especially distinguished. Thus, augmented reality technology has found its application in many different fields. The article discusses the general concepts and principles of building augmented reality systems.

Recent technological advances in developing intelligent telecommunication networks, ultra-compact bendable wireless transceiver chips, adaptive wearable sensors and actuators, and secure computing infrastructures along with the progress made in psychology and neuroscience for understanding neu-rocognitive and computational principles of human behavior combined have paved the way for a new field of research: Tactile Internet with Human-in-the-Loop (TaHiL). This emerging field of transdisciplinary research aims to promote next generation digitalized human-machine interactions in perceived real time. To achieve this goal, mechanisms and principles of human goal-directed multisensory perception and action need to be integrated into technological designs for breakthrough innovations in mobile telecommunication, electronics and materials engineering, as well as computing. This overview highlights key challenges and the frontiers of research in the new field of TaHiL. Revolutionizing the current Internet as a digital infrastructure for sharing visual and auditory information globally, the TaHiL research will enable humans to share tactile and haptic information and thus veridically immerse themselves into virtual, remote, or inaccessible real environments to exchange skills and expertise with other humans or machines for applications in medicine, industry, and the Internet of Skills.

With the maturity of Internet of things technology, massive data transmission has become the focus of research. In order to solve the problem of low speed of traditional hybrid data fast distribution algorithm for wireless sensor networks, a hybrid data fast distribution algorithm for wireless sensor networks based on visual Internet of things is designed. The logic structure of mixed data input gate in wireless sensor network is designed through the visual Internet of things. The objective function of fast distribution of mixed data in wireless sensor network is proposed. The number of copies of data to be distributed is dynamically calculated and the message deletion strategy is determined. Then the distribution parameters are calibrated, and the fitness ranking is performed according to the distribution quantity to complete the algorithm design. The experimental results show that the distribution rate of the designed algorithm is significantly higher than that of the control group, which can solve the problem of low speed of traditional data fast distribution algorithm.

An image hiding algorithm based on bit plane and two-dimensional code is proposed in this paper. The main characteristic of information hiding is to use the information redundant data of the existing image, to embed the information into these redundant data by the information hiding algorithm, or to partially replace redundant information with information to be embedded to achieve a visual invisible purpose. We first analyze the color index usage frequency of the block index matrix in the algorithm, and calculate the distance between the color of the block index matrix with only one color and the other color in the palette that is closest to the color. Then, the QR model and the compression model are applied to improve the efficiency. We compare the proposed model with the stateof-the-art models.

The network output activation values for a given input can be employed to produce a sorted ranking. Adversarial attacks typically generate the least amount of perturbation required to change the classifier label. In that sense, generated adversarial attack perturbation only affects the output in the 1st sorted ranking position. We argue that meaningful information about the adversarial examples i.e., their original labels, is still encoded in the network output ranking and could potentially be extracted, using rule-based reasoning. To this end, we introduce a novel adversarial attack methodology inspired by the K-anonymity principles, that generates adversarial examples that are not only misclassified, but their output sorted ranking spreads uniformly along K different positions. Any additional perturbation arising from the strength of the proposed objectives, is regularized by a visual similarity-based term. Experimental results denote that the proposed approach achieves the optimization goals inspired by K-anonymity with reduced perturbation as well.

A kitchen underpinned by the Internet of Things (IoT) requires the management of complex procedural processes. This is due to the fact that when supporting an end-user in the preparation of even only one dish, various devices may need to coordinate with each other. Additionally, it is challenging— yet desirable—to enable an end-user to program their kitchen devices according to their preferred behaviour and to allow them to visualise and track their cooking workflows. In this paper, we compared two semantic representations, namely, Behaviour Trees and the DX-MAN model. We analysed these representations based on their suitability for a range of end-users (i.e., novice to experienced). The methodology required the analysis of smart kitchen user requirements, from which we inferred that the main architectural requirements for IoT cooking workflows are variability and compositionality. Guided by the user requirements, we examined various scenarios and analysed workflow complexity and feasibility for each representation. On the one hand, we found that execution complexity tends to be higher on Behaviour Trees. However, due to their fallback node, they provide more transparency on how to recover from unprecedented circumstances. On the other hand, parameter complexity tends to be somewhat higher for the DX-MAN model. Nevertheless, the DX-MAN model can be favourable due to its compositionality aspect and the ease of visualisation it can offer.

Neural Module Network (NMN) exhibits strong interpretability and compositionality thanks to its handcrafted neural modules with explicit multi-hop reasoning capability. However, most NMNs suffer from two critical draw-backs: 1) scalability: customized module for specific function renders it impractical when scaling up to a larger set of functions in complex tasks; 2) generalizability: rigid pre-defined module inventory makes it difficult to generalize to unseen functions in new tasks/domains. To design a more powerful NMN architecture for practical use, we propose Meta Module Network (MMN) centered on a novel meta module, which can take in function recipes and morph into diverse instance modules dynamically. The instance modules are then woven into an execution graph for complex visual reasoning, inheriting the strong explainability and compositionality of NMN. With such a flexible instantiation mechanism, the parameters of instance modules are inherited from the central meta module, retaining the same model complexity as the function set grows, which promises better scalability. Meanwhile, as functions are encoded into the embedding space, unseen functions can be readily represented based on its structural similarity with previously observed ones, which ensures better generalizability. Experiments on GQA and CLEVR datasets validate the superiority of MMN over state-of-the-art NMN designs. Synthetic experiments on held-out unseen functions from GQA dataset also demonstrate the strong generalizability of MMN. Our code and model are released in Github1.

Naval vessels infrastructures are evolving towards increasingly connected and automatic systems. Such accelerated complexity boost to search for more adapted and useful navigation devices may be at odds with cybersecurity, making necessary to develop adapted analysis solutions for experts. This paper introduces a novel process to visualize and analyze naval Cyber-Physical Systems (CPS) using oriented graphs, considering operational constraints, to represent physical and functional connections between multiple components of CPS. Rapid prototyping of interconnected components is implemented in a semi-automatic manner by defining the CPS’s digital and physical systems as nodes, along with system variables as edges, to form three layers of an oriented graph, using the open-source Neo4j software suit. The generated multi-layer graph can be used to support cybersecurity analysis, like attacks simulation, anomaly detection and propagation estimation, applying existing or new algorithms.

The continuous increase in sophistication of threat actors over the years has made the use of actionable threat intelligence a critical part of the defence against them. Such Cyber Threat Intelligence is published daily on several online sources, including vulnerability databases, CERT feeds, and social media, as well as on forums and web pages from the Surface and the Dark Web. Named Entity Recognition (NER) techniques can be used to extract the aforementioned information in an actionable form from such sources. In this paper we investigate how the latest advances in the NER domain, and in particular transformer-based models, can facilitate this process. To this end, the dataset for NER in Threat Intelligence (DNRTI) containing more than 300 pieces of threat intelligence reports from open source threat intelligence websites is used. Our experimental results demonstrate that transformer-based techniques are very effective in extracting cybersecurity-related named entities, by considerably outperforming the previous state- of-the-art approaches tested with DNRTI.

This prototype demonstrates the viability of manipulating both physical and virtual objects with the same tool in order to maintain object permanence across both modes of interaction. Using oppositional force feedback, provided by a servo, and an augmented visual interface, provided by the user’s smartphone, this tool simulates the look and feel of a physical object within an augmented environment. Additionally, the tool is also able to manipulate physical objects that are not part of the augmented reality, such as a physical nut. By integrating both modes of interaction into the same tool, users can fluidly move between these different modes of interaction, manipulating both physical and virtual objects as the need arises. By overlaying this kind of visual and haptic augmentation onto a common tool such as a pair of pliers, we hope to further explore scenarios for collaborative telepresence in future work.