Biblio

Anomaly detection for devices (e.g, sensors and actuators) plays a crucial role in Industrial Control Systems (ICS) for security protection. The typical framework of deep learning-based anomaly detection includes a model to predict or reconstruct the state of devices and a detection mechanism to determine anomalies. The majority of anomaly detection methods use a fixed threshold detection mechanism to detect anomalous points. However, the anomalies caused by cyberattacks in ICSs are usually continuous anomaly segments. In this paper, we propose a novel detection mechanism to detect continuous anomaly segments. Its core idea is to determine the start and end times of anomalies based on the continuity characteristics of anomalies and the dynamics of error. We conducted experiments on the two real-world datasets for performance evaluation using five baselines. The F1 score increased by 3.8% on average in the SWAT dataset and increased by 15.6% in the WADI dataset. The results show a significant improvement in the performance of baselines using an error neighborhood-based continuity detection mechanism in a real-time manner.

In recent years, generative adversarial networks (GAN) have become a research hotspot in the field of deep learning. Researchers apply them to the field of anomaly detection and are committed to effectively and accurately identifying abnormal images in practical applications. In anomaly detection, traditional supervised learning algorithms have limitations in training with a large number of known labeled samples. Therefore, the anomaly detection model of unsupervised learning GAN is the research object for discussion and research. Firstly, the basic principles of GAN are introduced. Secondly, several typical GAN-based anomaly detection models are sorted out in detail. Then by comparing the similarities and differences of each derivative model, discuss and summarize their respective advantages, limitations and application scenarios. Finally, the problems and challenges faced by GAN in anomaly detection are discussed, and future research directions are prospected.

Anomaly detection has been considered under several extents of prior knowledge. Unsupervised methods do not require any labelled data, whereas semi-supervised methods leverage some known anomalies. Inspired by mixture-of-experts models and the analysis of the hidden activations of neural networks, we introduce a novel data-driven anomaly detection method called ARGUE. Our method is not only applicable to unsupervised and semi-supervised environments, but also profits from prior knowledge of self-supervised settings. We designed ARGUE as a combination of dedicated expert networks, which specialise on parts of the input data. For its final decision, ARGUE fuses the distributed knowledge across the expert systems using a gated mixture-of-experts architecture. Our evaluation motivates that prior knowledge about the normal data distribution may be as valuable as known anomalies.

Increasing connectivity and automation in vehicles leads to a greater potential attack surface. Such vulnerabilities within vehicles can also be used for auto-theft, increasing the potential for attackers to disable anti-theft mechanisms implemented by vehicle manufacturers. We utilize patterns derived from Controller Area Network (CAN) bus traffic to verify driver “behavior”, as a basis to prevent vehicle theft. Our proposed model uses semi-supervised learning that continuously profiles a driver, using features extracted from CAN bus traffic. We have selected 15 key features and obtained an accuracy of 99% using a dataset comprising a total of 51 features across 10 different drivers. We use a number of data analysis algorithms, such as J48, Random Forest, JRip and clustering, using 94K records. Our results show that J48 is the best performing algorithm in terms of training and testing (1.95 seconds and 0.44 seconds recorded, respectively). We also analyze the effect of using a sliding window on algorithm performance, altering the size of the window to identify the impact on prediction accuracy.

With the rapid development of communication net-work, the types and quantities of network traffic data have in-creased substantially. What followed was the frequent occurrence of versatile cyber attacks. As an important part of network security, the network-based intrusion detection system (NIDS) can monitor and protect the network equippments and terminals in real time. The traditional detection methods based on deep learning (DL) are always in supervised manners in NIDS, which can automatically build end-to-end detection model without man-ual feature extraction and selection by domain experts. However, supervised learning methods require large-scale labeled data, yet capturing large labeled datasets is a very cubersome, tedious and time-consuming manual task. Instead, unsupervised learning is an effective way to overcome this problem. Nonetheless, the ex-isting unsupervised methods are prone to low detection efficiency and are difficult to train. In this paper we propose a novel NIDS method called PGAN based on generative adversarial network (GAN) to detect the abnormal traffic from the perspective of Anomaly Detection, which leverage the competitive speciality of adversarial training to learn the normal traffic. Based on the public dataset CICIDS2017, three experimental results show that PGAN can significantly outperform other unsupervised methods like stacked autoencoder (SAE) and isolation forest (IF).

Data is one of the most valuable assets of an organization and has a tremendous impact on its long-term success and decision-making processes. Typically, organizational data error and outlier detection processes perform manually and reactively, making them time-consuming and prone to human errors. Additionally, rich data types, unlabeled data, and increased volume have made such data more complex. Accordingly, an automated anomaly detection approach is required to improve data management and quality control processes. This study introduces an unsupervised anomaly detection approach based on models comparison, consensus learning, and a combination of rules of thumb with iterative hyper-parameter tuning to increase data quality. Furthermore, a domain expert is considered a human in the loop to evaluate and check the data quality and to judge the output of the unsupervised model. An experiment has been conducted to assess the proposed approach in the context of a case study. The experiment results confirm that the proposed approach can improve the quality of organizational data and facilitate anomaly detection processes.

In recent years, insider threat incidents and losses of companies or organizations are on the rise, and internal network security is facing great challenges. Traditional intrusion detection methods cannot identify malicious behaviors of insiders. As an effective method, insider threat detection technology has been widely concerned and studied. In this paper, we use the tree structure method to analyze user behavior, form feature sequences, and combine the Copula Based Outlier Detection (COPOD) method to detect the difference between feature sequences and identify abnormal users. We experimented on the insider threat dataset CERT-IT and compared it with common methods such as Isolation Forest.

Due to immense need for implementing security measures and control ongoing activities, intelligent video analytics is regarded as one of the outstanding and challenging research domains in Computer Vision. Assigning video operator to manually monitor the surveillance videos 24×7 to identify occurrence of interesting and anomalous events like robberies, wrong U-turns, violence, accidents is cumbersome and error- prone. Therefore, to address the issue of continuously monitoring surveillance videos and detect the anomalies from them, a deep learning approach based on pipelined sequence of convolutional autoencoder and sequence to sequence long short-term memory autoencoder has been proposed. Specifically, unsupervised learning approach encompassing one-class classification paradigm has been proposed for detection of anomalies in videos. The effectiveness of the propped model is demonstrated on benchmarked anomaly detection dataset and significant results in terms of equal error rate, area under curve and time required for detection have been achieved.

Most machine learning-based malware detection systems use various supervised learning methods to classify different instances of software as benign or malicious. This approach provides no information regarding the behavioral characteristics of malware. It also requires a large amount of training data and is prone to labeling difficulties and can reduce accuracy due to redundant training data. Therefore, we propose a malware detection method based on deep learning, which uses malware images and a set of autoencoders to detect malware. The method is to design an autoencoder to learn the functional characteristics of malware, and then to observe the reconstruction error of autoencoder to realize the classification and detection of malware and benign software. The proposed approach achieves 93% accuracy and comparatively better F1-score values while detecting malware and needs little training data when compared with traditional malware detection systems.

To harness the value of data generated from IoT, there is a crucial requirement of new mechanisms. Machine learning (ML) is among the most suitable paradigms of computation which embeds strong intelligence within IoT devices. Various ML techniques are being widely utilised for improving network security in IoT. These techniques include reinforcement learning, semi-supervised learning, supervised learning, and unsupervised learning. This report aims to critically analyse the role played by supervised and unsupervised ML for the enhancement of IoT security.

With current IoT architectures, once a single device in a network is compromised, it can be used to disrupt the behavior of other devices on the same network. Even though system administrators can secure critical devices in the network using best practices and state-of-the-art technology, a single vulnerable device can undermine the security of the entire network. The goal of this work is to limit the ability of an attacker to exploit a vulnerable device on an IoT network and fabricate deceitful messages to co-opt other devices. The approach is to limit attackers by using device proxies that are used to retransmit and control network communications. We present an architecture that prevents deceitful messages generated by compromised devices from affecting the rest of the network. The design assumes a centralized and trustworthy machine that can observe the behavior of all devices on the network. The central machine collects application layer data, as opposed to low-level network traffic, from each IoT device. The collected data is used to train models that capture the normal behavior of each individual IoT device. The normal behavioral data is then used to monitor the IoT devices and detect anomalous behavior. This paper reports on our experiments using both a binary classifier and a density-based clustering algorithm to model benign IoT device behavior with a realistic test-bed, designed to capture normal behavior in an IoT-monitored environment. Results from the IoT testbed show that both the classifier and the clustering algorithms are promising and encourage the use of application-level data for detecting compromised IoT devices.

To ensure quality of service and user experience, large Internet companies often monitor various Key Performance Indicators (KPIs) of their systems so that they can detect anomalies and identify failure in real time. However, due to a large number of various KPIs and the lack of high-quality labels, existing KPI anomaly detection approaches either perform well only on certain types of KPIs or consume excessive resources. Therefore, to realize generic and practical KPI anomaly detection in the real world, we propose a KPI anomaly detection framework named iRRCF-Active, which contains an unsupervised and white-box anomaly detector based on Robust Random Cut Forest (RRCF), and an active learning component. Specifically, we novelly propose an improved RRCF (iRRCF) algorithm to overcome the drawbacks of applying original RRCF in KPI anomaly detection. Besides, we also incorporate the idea of active learning to make our model benefit from high-quality labels given by experienced operators. We conduct extensive experiments on a large-scale public dataset and a private dataset collected from a large commercial bank. The experimental resulta demonstrate that iRRCF-Active performs better than existing traditional statistical methods, unsupervised learning methods and supervised learning methods. Besides, each component in iRRCF-Active has also been demonstrated to be effective and indispensable.

Substantial empirical evidence, including the success of synthetic graph generation models as well as of analytical methodologies, suggests that large, real graphs have a recursive community structure. The latter results, in part at least, in other important properties of these graphs such as low diameter, high clustering coefficient values, heavy degree distribution tail, and clustered graph spectrum. Notice that this structure need not be official or moderated like Facebook groups, but it can also take an ad hoc and unofficial form depending on the functionality of the social network under study as for instance the follow relationship on Twitter or the connections between news aggregators on Reddit. Community discovery is paramount in numerous applications such as political campaigns, digital marketing, crowdfunding, and fact checking. Here a tensor representation for Twitter subgraphs is proposed which takes into consideration both the followfollower relationships but also the coherency in hashtags. Community structure discovery then reduces to the computation of Tucker tensor decomposition, a higher order counterpart of the well-known unsupervised learning method of singular value decomposition (SVD). Tucker decomposition clearly outperforms the SVD in terms of finding a more compact community size distribution in experiments done in Julia on a Twitter subgraph. This can be attributed to the facts that the proposed methodology combines both structural and functional Twitter elements and that hashtags carry an increased semantic weight in comparison to ordinary tweets.

Detection of multi-stage threats such as Advanced Persistent Threats (APT) is extremely challenging due to their deceptive approaches. Sequential events of threats might look benign when performed individually or from different addresses. We propose a new unsupervised framework to identify patterns and correlations of malicious behaviours by analysing heterogeneous log-files. The framework consists of two main phases of data analysis to extract inner-behaviours of log-files and then the patterns of those behaviours over analysed files. To evaluate the framework we have produced a (publicly available) labelled version of the SotM43 dataset. Our results demonstrate that the framework can (i) efficiently cluster inner-behaviours of log-files with high accuracy and (ii) extract patterns of malicious behaviour and correlations between those patterns from real-world data.

This work uses adversarial perturbations to enhance deepfake images and fool common deepfake detectors. We created adversarial perturbations using the Fast Gradient Sign Method and the Carlini and Wagner L2 norm attack in both blackbox and whitebox settings. Detectors achieved over 95% accuracy on unperturbed deepfakes, but less than 27% accuracy on perturbed deepfakes. We also explore two improvements to deep-fake detectors: (i) Lipschitz regularization, and (ii) Deep Image Prior (DIP). Lipschitz regularization constrains the gradient of the detector with respect to the input in order to increase robustness to input perturbations. The DIP defense removes perturbations using generative convolutional neural networks in an unsupervised manner. Regularization improved the detection of perturbed deepfakes on average, including a 10% accuracy boost in the blackbox case. The DIP defense achieved 95% accuracy on perturbed deepfakes that fooled the original detector while retaining 98% accuracy in other cases on a 100 image subsample.

Intrusion detection system based on representation learning is the main research direction in the field of anomaly detection. Malicious traffic detection system can distinguish normal and malicious traffic by learning representations between normal and malicious traffic. However, under the context of big data, there are many types of malicious traffic, and the features are also changing constantly. It is still a urgent problem to design a detection model that can effectively learn and summarize the feature of normal traffic and accurately identify the features of new kinds of malicious traffic.in this paper, a malicious traffic detection method based on Deep Support Vector Data Description is proposed, which is called Deep - SVDD. We combine convolutional neural network (CNN) with support vector data description, and train the model with normal traffic. The normal traffic features are mapped to high-dimensional space through neural networks, and a compact hypersphere is trained by unsupervised learning, which includes the normal features of the highdimensional space. Malicious traffic fall outside the hypersphere, thus distinguishing between normal and malicious traffic. Experiments show that the model has a high detection rate and a low false alarm rate, and it can effectively identify new malicious traffic.

The rapid growth of Android malware has posed severe security threats to smartphone users. On the basis of the familial trait of Android malware observed by previous work, the familial analysis is a promising way to help analysts better focus on the commonalities of malware samples within the same families, thus reducing the analytical workload and accelerating malware analysis. The majority of existing approaches rely on supervised learning and face three main challenges, i.e., low accuracy, low efficiency, and the lack of labeled dataset. To address these challenges, we first construct a fine-grained behavior model by abstracting the program semantics into a set of subgraphs. Then, we propose SRA, a novel feature that depicts the similarity relationships between the Structural Roles of sensitive API call nodes in subgraphs. An SRA is obtained based on graph embedding techniques and represented as a vector, thus we can effectively reduce the high complexity of graph matching. After that, instead of training a classifier with labeled samples, we construct malware link network based on SRAs and apply community detection algorithms on it to group the unlabeled samples into groups. We implement these ideas in a system called GefDroid that performs Graph embedding based familial analysis of AnDroid malware using unsupervised learning. Moreover, we conduct extensive experiments to evaluate GefDroid on three datasets with ground truth. The results show that GefDroid can achieve high agreements (0.707-0.883 in term of NMI) between the clustering results and the ground truth. Furthermore, GefDroid requires only linear run-time overhead and takes around 8.6s to analyze a sample on average, which is considerably faster than the previous work.

The Internet of Vehicles (IoV) will connect not only mobile devices with vehicles, but it will also connect vehicles with each other, and with smart offices, buildings, homes, theaters, shopping malls, and cities. The IoV facilitates optimal and reliable communication services to connected vehicles in smart cities. The backbone of connected vehicles communication is the critical V2X infrastructures deployment. The spectrum utilization depends on the demand by the end users and the development of infrastructure that includes efficient automation techniques together with the Internet of Things (IoT). The infrastructure enables us to build smart environments for spectrum utilization, which we refer to as Smart Spectrum Utilization (SSU). This paper presents an integrated system consisting of SSU with IoV. However, the tasks of securing IoV and protecting it from cyber attacks present considerable challenges. This paper introduces an IoV security system using deep learning approach to develop secure applications and reliable services. Deep learning composed of unsupervised learning and supervised learning, could optimize the IoV security system. The deep learning methodology is applied to monitor security threats. Results from simulations show that the monitoring accuracy of the proposed security system is superior to that of the traditional system.

With the proposal of the national industrial 4.0 strategy, the integration of industrial control network and Internet technology is getting higher and higher. At the same time, the closeness of industrial control networks has been broken to a certain extent, making the problem of industrial control network security increasingly serious. S7 protocol is a private protocol of Siemens Company in Germany, which is widely used in the communication process of industrial control network. In this paper, an industrial control intrusion detection model based on S7 protocol is proposed. Traditional protocol parsing technology cannot resolve private industrial control protocols, so, this model uses deep analysis algorithm to realize the analysis of S7 data packets. At the same time, in order to overcome the complexity and portability of static white list configuration, this model dynamically builds a white list through white list self-learning algorithm. Finally, a composite intrusion detection method combining white list detection and abnormal behavior detection is used to detect anomalies. The experiment proves that the method can effectively detect the abnormal S7 protocol packet in the industrial control network.

Modern infrastructure is heavily reliant on systems with interconnected computational and physical resources, named Cyber-Physical Systems (CPSs). Hence, building resilient CPSs is a prime need and continuous monitoring of the CPS operational health is essential for improving resilience. This paper presents a framework for calculating and monitoring of health in CPSs using data driven techniques. The main advantages of this data driven methodology is that the ability of leveraging heterogeneous data streams that are available from the CPSs and the ability of performing the monitoring with minimal a priori domain knowledge. The main objective of the framework is to warn the operators of any degradation in cyber, physical or overall health of the CPS. The framework consists of four components: 1) Data acquisition and feature extraction, 2) state identification and real time state estimation, 3) cyber-physical health calculation and 4) operator warning generation. Further, this paper presents an initial implementation of the first three phases of the framework on a CPS testbed involving a Microgrid simulation and a cyber-network which connects the grid with its controller. The feature extraction method and the use of unsupervised learning algorithms are discussed. Experimental results are presented for the first two phases and the results showed that the data reflected different operating states and visualization techniques can be used to extract the relationships in data features.

In this paper we propose a responsive autonomic and data-driven adaptive virtual networking framework (RAvN) that integrates the adaptive reconfigurable features of a popular SDN platform called open networking operating system (ONOS), the network performance statistics provided by traffic monitoring tools such as T-shark or sflow-RT and analytics and decision making skills provided from new and current machine learning techniques to detect and mitigate anomalous behavior. For this paper we focus on the development of novel detection schemes using a developed Centroid-based clustering technique and the Intragroup variance of data features within network traffic (C. Intra), with a multivariate gaussian distribution model fitted to the constant changes in the IP addresses of the network to accurately assist in the detection of low rate and high rate denial of service (DoS) attacks. We briefly discuss our ideas on the development of the decision-making and execution component using the concept of generating adaptive policy updates (i.e. anomalous mitigation solutions) on-the-fly to the ONOS SDN controller for updating network configurations and flows. In addition we provide the analysis on anomaly detection schemes used for detecting low rate and high rate DoS attacks versus a commonly used unsupervised machine learning technique Kmeans. The proposed schemes outperformed Kmeans significantly. The multivariate clustering method and the intragroup variance recorded 80.54% and 96.13% accuracy respectively while Kmeans recorded 72.38% accuracy.

This paper presents TrustSign, a novel, trusted automatic malware signature generation method based on high-level deep features transferred from a VGG-19 neural network model pre-trained on the ImageNet dataset. While traditional automatic malware signature generation techniques rely on static or dynamic analysis of the malware's executable, our method overcomes the limitations associated with these techniques by producing signatures based on the presence of the malicious process in the volatile memory. Signatures generated using TrustSign well represent the real malware behavior during runtime. By leveraging the cloud's virtualization technology, TrustSign analyzes the malicious process in a trusted manner, since the malware is unaware and cannot interfere with the inspection procedure. Additionally, by removing the dependency on the malware's executable, our method is capable of signing fileless malware. Thus, we focus our research on in-browser cryptojacking attacks, which current antivirus solutions have difficulty to detect. However, TrustSign is not limited to cryptojacking attacks, as our evaluation included various ransomware samples. TrustSign's signature generation process does not require feature engineering or any additional model training, and it is done in a completely unsupervised manner, obviating the need for a human expert. Therefore, our method has the advantage of dramatically reducing signature generation and distribution time. The results of our experimental evaluation demonstrate TrustSign's ability to generate signatures invariant to the process state over time. By using the signatures generated by TrustSign as input for various supervised classifiers, we achieved 99.5% classification accuracy.

Cryptojacking is the permissionless use of a target device to covertly mine cryptocurrencies. With cryptojacking attackers use malicious JavaScript codes to force web browsers into solving proof-of-work puzzles, thus making money by exploiting resources of the website visitors. To understand and counter such attacks, we systematically analyze the static, dynamic, and economic aspects of in-browser cryptojacking. For static analysis, we perform content-, currency-, and code-based categorization of cryptojacking samples to 1) measure their distribution across websites, 2) highlight their platform affinities, and 3) study their code complexities. We apply unsupervised learning to distinguish cryptojacking scripts from benign and other malicious JavaScript samples with 96.4% accuracy. For dynamic analysis, we analyze the effect of cryptojacking on critical system resources, such as CPU and battery usage. Additionally, we perform web browser fingerprinting to analyze the information exchange between the victim node and the dropzone cryptojacking server. We also build an analytical model to empirically evaluate the feasibility of cryptojacking as an alternative to online advertisement. Our results show a large negative profit and loss gap, indicating that the model is economically impractical. Finally, by leveraging insights from our analyses, we build countermeasures for in-browser cryptojacking that improve upon the existing remedies.

Recently, hashing has attracted considerable attention for nearest neighbor search due to its fast query speed and low storage cost. However, existing unsupervised hashing algorithms have two problems in common. Firstly, the widely utilized anchor graph construction algorithm has inherent limitations in local weight estimation. Secondly, the locally linear structure in the original feature space is seldom taken into account for binary encoding. Therefore, in this paper, we propose a novel unsupervised hashing method, dubbed “discrete locally-linear preserving hashing”, which effectively calculates the adjacent matrix while preserving the locally linear structure in the obtained hash space. Specifically, a novel local anchor embedding algorithm is adopted to construct the approximate adjacent matrix. After that, we directly minimize the reconstruction error with the discrete constrain to learn the binary codes. Experimental results on two typical image datasets indicate that the proposed method significantly outperforms the state-of-the-art unsupervised methods.

Despite the popularity of wireless sensor networks (WSNs) in a wide range of applications, security problems associated with them have not been completely resolved. Middleware is generally introduced as an intermediate layer between WSNs and the end user to resolve some limitations, but most of the existing middleware is unable to protect data from malicious and unknown attacks during transmission. This paper introduces an intelligent middleware based on an unsupervised learning technique called Generative Adversarial Networks (GANs) algorithm. GANs contain two networks: a generator (G) network and a detector (D) network. The G creates fake data similar to the real samples and combines it with real data from the sensors to confuse the attacker. The D contains multi-layers that have the ability to differentiate between real and fake data. The output intended for this algorithm shows an actual interpretation of the data that is securely communicated through the WSN. The framework is implemented in Python with experiments performed using Keras. Results illustrate that the suggested algorithm not only improves the accuracy of the data but also enhances its security by protecting data from adversaries. Data transmission from the WSN to the end user then becomes much more secure and accurate compared to conventional techniques.