Biblio

In order to realize Intelligent Disaster Recovery and break the traditional reactive backup mode, it is necessary to forecast the potential system anomalies, and proactively backup the real-time datas and configurations. System logs record the running status as well as the critical events (including errors and warnings), which can help to detect system performance, debug system faults and analyze the causes of anomalies. What's more, with the features of real-time, hierarchies and easy-access, log data can be an ideal source for monitoring system status. To reduce the complexity and improve the robustness and practicability of existing log-based anomaly detection methods, we propose a new anomaly detection mechanism based on hierarchical weights, which can deal with unstable log data. We firstly extract semantic information of log strings, and get the word-level weights by SIF algorithm to embed log strings into vectors, which are then feed into attention-based Long Short-Term Memory(LSTM) deep learning network model. In addition to get sentence-level weight which can be used to explore the interdependence between different log sequences and improve the accuracy, we utilize attention weights to help with building workflow to diagnose the abnormal points in the execution of a specific task. Our experimental results show that the hierarchical weights mechanism can effectively improve accuracy of perdition task and reduce complexity of the model, which provides the feasibility foundation support for Intelligent Disaster Recovery.

Autonomous cyber-physical systems (CPS) are susceptible to non-invasive physical attacks such as sensor spoofing attacks that are beyond the classical cybersecurity domain. These attacks have motivated numerous research efforts on attack detection, but little attention on what to do after detecting an attack. The importance of attack recovery is emphasized by the need to mitigate the attack’s impact on a system and restore it to continue functioning. There are only a few works addressing attack recovery, but they all rely on prior knowledge of system dynamics. To overcome this limitation, we propose Recovery-by-Learning, a data-driven attack recovery framework that restores CPS from sensor attacks. The framework leverages natural redundancy among heterogeneous sensors and historical data for attack recovery. Specially, the framework consists of two major components: state predictor and data checkpointer. First, the predictor is triggered to estimate systems states after the detection of an attack. We propose a deep learning-based prediction model that exploits the temporal correlation among heterogeneous sensors. Second, the checkpointer executes when no attack is detected. We propose a double sliding window based checkpointing protocol to remove compromised data and keep trustful data as input to the state predictor. Third, we implement and evaluate the effectiveness of our framework using a realistic data set and a ground vehicle simulator. The results show that our method restores a system to continue functioning in presence of sensor attacks.

Deep neural networks (DNNs) have gained significant popularity in recent years, becoming the state of the art in a variety of domains. In particular, deep reinforcement learning (DRL) has recently been employed to train DNNs that realize control policies for various types of real-world systems. In this work, we present the whiRL 2.0 tool, which implements a new approach for verifying complex properties of interest for DRL systems. To demonstrate the benefits of whiRL 2.0, we apply it to case studies from the communication networks domain that have recently been used to motivate formal verification of DRL systems, and which exhibit characteristics that are conducive for scalable verification. We propose techniques for performing k-induction and semi-automated invariant inference on such systems, and leverage these techniques for proving safety and liveness properties that were previously impossible to verify due to the scalability barriers of prior approaches. Furthermore, we show how our proposed techniques provide insights into the inner workings and the generalizability of DRL systems. whiRL 2.0 is publicly available online.

The Supervisory control and data acquisition (SCADA) systems have been continuously leveraging the evolution of network architecture, communication protocols, next-generation communication techniques (5G, 6G, Wi-Fi 6), and the internet of things (IoT). However, SCADA system has become the most profitable and alluring target for ransomware attackers. This paper proposes the deep learning-based novel ransomware detection framework in the SCADA controlled electric vehicle charging station (EVCS) with the performance analysis of three deep learning algorithms, namely deep neural network (DNN), 1D convolution neural network (CNN), and long short-term memory (LSTM) recurrent neural network. All three-deep learning-based simulated frameworks achieve around 97% average accuracy (ACC), more than 98% of the average area under the curve (AUC) and an average F1-score under 10-fold stratified cross-validation with an average false alarm rate (FAR) less than 1.88%. Ransomware driven distributed denial of service (DDoS) attack tends to shift the state of charge (SOC) profile by exceeding the SOC control thresholds. Also, ransomware driven false data injection (FDI) attack has the potential to damage the entire BES or physical system by manipulating the SOC control thresholds. It's a design choice and optimization issue that a deep learning algorithm can deploy based on the tradeoffs between performance metrics.

Supervisory control and data acquisition (SCADA) networks provide high situational awareness and automation control for industrial control systems, whilst introducing a wide range of access points for cyber attackers. To address these issues, a line of machine learning or deep learning based intrusion detection systems (IDSs) have been presented in the literature, where a large number of attack examples are usually demanded. However, in real-world SCADA networks, attack examples are not always sufficient, having only a few shots in many cases. In this paper, we propose a novel few-shot learning based IDS, named FS-IDS, to detect cyber attacks against SCADA networks, especially when having only a few attack examples in the defenders’ hands. Specifically, a new method by orchestrating one-hot encoding and principal component analysis is developed, to preprocess SCADA datasets containing sufficient examples for frequent cyber attacks. Then, a few-shot learning based preliminary IDS model is designed and trained using the preprocessed data. Last, a complete FS-IDS model for SCADA networks is established by further training the preliminary IDS model with a few examples for cyber attacks of interest. The high effectiveness of the proposed FS-IDS, in detecting cyber attacks against SCADA networks with only a few examples, is demonstrated by extensive experiments on a real SCADA dataset.

Supervisory control and data acquisition (SCADA) systems are used with monitoring and control purposes for the process not to fail in industrial control systems. Today, the increase in the use of standard protocols, hardware, and software in the SCADA systems that can connect to the internet and institutional networks causes these systems to become a target for more cyber-attacks. Intrusion detection systems are used to reduce or minimize cyber-attack threats. The use of deep learning-based intrusion detection systems also increases in parallel with the increase in the amount of data in the SCADA systems. The unsupervised feature learning present in the deep learning approaches enables the learning of important features within the large datasets. The features learned in an unsupervised way by using deep learning techniques are used in order to classify the data as normal or abnormal. Architectures such as convolutional neural network (CNN), Autoencoder (AE), deep belief network (DBN), and long short-term memory network (LSTM) are used to learn the features of SCADA data. These architectures use softmax function, extreme learning machine (ELM), deep belief networks, and multilayer perceptron (MLP) in the classification process. In this study, anomaly-based intrusion detection systems consisting of convolutional neural network, autoencoder, deep belief network, long short-term memory network, or various combinations of these methods on the SCADA networks in the literature were analyzed and the positive and negative aspects of these approaches were explained through their attack detection performances.

Natural language processing (NLP) is a division of artificial intelligence. The constructed model's quality is entirely reliant on the training dataset's quality. A data streaming pipeline is an adhesive application, completing a managed connection from data sources to machine learning methods. The recommended NLP pipeline composition has well-defined procedures. The implemented message broker design is a usual apparatus for delivering events. It makes it achievable to construct a robust training dataset for machine learning use-case and serve the model's input. The reconstructed dataset is a valid input for the machine learning processes. Based on the data pipeline's product, the model recreation and redeployment can be scheduled automatically.

In today's world, there is rapid progress in the field of artificial intelligence and image captioning. It becomes a fascinating task that has saw widespread interest. The task of image captioning comprises image description engendered based on the hybrid combination of deep learning, natural language processing, and various approaches of machine learning and computer vision. In this work authors emphasize on how the model generates a short description as an output of the input image using the functionalities of Deep Learning and Natural Language Processing, for helping visually impaired people, and can also be cast-off in various web sites to automate the generation of captions reducing the task of recitation with great ease.

Style transfer means using a neural network to extract the content of one image and the style of the other image. The two are combined to get the final result, broadly applied in social communication, animation production, entertainment items. Using style transfer, users can share and exchange images; painters can create specific art styles more readily with less creation cost and production time. Therefore, style transfer is widely concerned recently due to its various and valuable applications. In the past few years, the paper reviews style transfer and chooses three representative works to analyze in detail and contrast with each other, including StyleGAN, CycleGAN, and TL-GAN. Moreover, what function an ideal model of style transfer should realize is discussed. Compared with such a model, potential problems and prospects of different methods to achieve style transfer are listed. A couple of solutions to these drawbacks are given in the end.

Art is the perfect way for people to express their emotions in a way that words are unable to do. By simply looking at art, we can understand a person’s creativity and thoughts. In former times, artists spent a great deal of time creating an image of varied styles. In the current deep learning era, we are able to create images of different styles as we prefer within a short period of time. Neural style transfer is the most popular and widely used deep learning application that applies the desired style to the content image, which in turn generates an output image that is a combination of both style and the content of the original image. In this paper we have implemented the neural style transfer model with two architectures namely Vgg19 and Alexnet. This paper compares the output-styled image and the total loss obtained through VGG19 and Alexnet architectures. In addition, three different activation functions are used to compare quality and total loss of output styled images within Alexnet architectures.

The goal of neural style transfer is to transform images by the deep learning method, such as changing oil paintings into sketch-style images. The Generative Adversarial Network (GAN) has made remarkable achievements in neural style transfer in recent years. At first, this paper introduces three typical neural style transfer methods, including StyleGAN, StarGAN, and Transparent Latent GAN (TL-GAN). Then, we discuss the advantages and disadvantages of these models, including the quality of the feature axis, the scale, and the model's interpretability. In addition, as the core of this paper, we put forward innovative improvements to the above models, including how to fully exploit the advantages of the above three models to derive a better style conversion model.

In recent years, the application of style transfer has become more and more widespread. Traditional deep learning-based style transfer networks often have problems such as image distortion, loss of detailed information, partial content disappearance, and transfer errors. The style transfer network based on deep learning that we propose in this article is aimed at dealing with these problems. Our method uses image edge information fusion and semantic segmentation technology to constrain the image structure before and after the migration, so that the converted image maintains structural consistency and integrity. We have verified that this method can successfully suppress image conversion distortion in most scenarios, and can generate good results.

In recent years, neural style transfer have shown impressive results in deep learning. In particular, for text style transfer, recent researches have successfully completed the transition from the text font domain to the text style domain. However, for text style transfer, multiple style transfer often requires learning many models, and generating multiple styles images of texts in a single model remains an unsolved problem. In this paper, we propose a multiple style transformation network for text style transfer, which can generate multiple styles of text images in a single model and control the style of texts in a simple way. The main idea is to add conditions to the transfer network so that all the styles can be trained effectively in the network, and to control the generation of each text style through the conditions. We also optimize the network so that the conditional information can be transmitted effectively in the network. The advantage of the proposed network is that multiple styles of text can be generated with only one model and that it is possible to control the generation of text styles. We have tested the proposed network on a large number of texts, and have demonstrated that it works well when generating multiple styles of text at the same time.

In order to detect the objects on the water surface, a neural style learning algorithm is proposed in this paper. The algorithm uses the Gram matrix of a pre-trained convolutional neural network to represent the style of the texture in the image, which is originally used for image style transfer. The objects on the water surface can be easily distinguished by the difference in their styles of the image texture. The algorithm is tested on the dataset of the Airbus Ship Detection Challenge on Kaggle. Compared to the other water surface object detection algorithms, the proposed algorithm has a good precision of 0.925 with recall equals to 0.86.

Ensuring the network security, resists the malicious traffic attacks as much as possible, and ensuring the network security, the Gated Recurrent Unit (GRU) and Convolutional Neural Network (CNN) are combined. Then, a Software Defined Networking (SDN) anomaly detection architecture is built and continuously optimized to ensure network security as much as possible and enhance the reliability of the detection architecture. The results show that the proposed network architecture can greatly improve the accuracy of detection, and its performance will be different due to the different number of CNN layers. When the two-layer CNN structure is selected, its performance is the best among all algorithms. Especially, the accuracy of GRU- CNN-2 is 98.7%, which verifies that the proposed method is effective. Therefore, under deep learning, the utilization of GRU- CNN to explore and optimize the SDN anomaly detection is of great significance to ensure information transmission security in the future.

Due to the wide deployment of deep learning applications in safety-critical systems, robust and secure execution of deep learning workloads is imperative. Adversarial examples, where the inputs are carefully designed to mislead the machine learning model is among the most challenging attacks to detect and defeat. The most dominant approach for defending against adversarial examples is to systematically create a network architecture that is sufficiently robust. Neural Architecture Search (NAS) has been heavily used as the de facto approach to design robust neural network models, by using the accuracy of detecting adversarial examples as a key metric of the neural network's robustness. While NAS has been proven effective in improving the robustness (and accuracy in general), the NAS-generated network models run noticeably slower on typical DNN accelerators than the hand-crafted networks, mainly because DNN accelerators are not optimized for robust NAS-generated models. In particular, the inherent multi-branch nature of NAS-generated networks causes unacceptable performance and energy overheads.To bridge the gap between the robustness and performance efficiency of deep learning applications, we need to rethink the design of AI accelerators to enable efficient execution of robust (auto-generated) neural networks. In this paper, we propose a novel hardware architecture, NASGuard, which enables efficient inference of robust NAS networks. NASGuard leverages a heuristic multi-branch mapping model to improve the efficiency of the underlying computing resources. Moreover, NASGuard addresses the load imbalance problem between the computation and memory-access tasks from multi-branch parallel computing. Finally, we propose a topology-aware performance prediction model for data prefetching, to fully exploit the temporal and spatial localities of robust NAS-generated architectures. We have implemented NASGuard with Verilog RTL. The evaluation results show that NASGuard achieves an average speedup of 1.74× over the baseline DNN accelerator.

In this paper, we propose a deep learning based detection for multiple input multiple output (MIMO) physical-layer network coding (DeepPNC) over two way relay channels (TWRC). In MIMO-PNC, the relay node receives the signals superimposed from the two end nodes. The relay node aims to obtain the network-coded (NC) form of the two end nodes' signals. By training suitable deep neural networks (DNNs) with a limited set of training samples. DeepPNC can extract the NC symbols from the superimposed signals received while the output of each layer in DNNs converges. Compared with the traditional detection algorithms, DeepPNC has higher mapping accuracy and does not require channel information. The simulation results show that the DNNs based DeepPNC can achieve significant gain over the DeepNC scheme and the other traditional schemes, especially when the channel matrix changes unexpectedly.

Modulated signal identification plays a crucial role in both military reconnaissance and civilian signal regulation. Traditionally, modulated signal identification is based on high-order statistics, but this approach has many drawbacks. With the development of deep learning, its advantages are fully exploited by combining it with modulated signals to avoid the complex process of computing a priori knowledge while having good fault tolerance. In this paper, ten digital modulated signals are classified and recognized, and improvements are made on the basis of convolutional neural networks, using feature reuse to increase the depth of the convolutional layer and extract signal features with better results. After experimental analysis, the recognition accuracy increases with the rise of the signal-to-noise ratio, and can reach 90% and above when the signal-to-noise ratio is 30dB.

Automatic modulation recognition (AMR) plays an important role in cognitive radio and electronic reconnaissance applications. In order to solve the problem that the lack of modulation signal data sets, the labeled data sets are generated by the software radio equipment NI-USRP 2920 and LabVIEW software development tool. In this paper, a combined network based on deep learning is proposed to identify ten types of digital modulation signals. Convolutional neural network (CNN) and Inception network are trained on different data sets, respectively. We combine CNN with Inception network to distinguish different modulation signals well. Experimental results show that our proposed method can recognize ten types of digital modulation signals with high identification accuracy, even in scenarios with a low signal-to-noise ratio (SNR).

With the ubiquitous network applications and the continuous development of network attack technology, all social circles have paid close attention to the cyberspace security. Intrusion detection systems (IDS) plays a very important role in ensuring computer and communication systems security. Recently, deep learning has achieved a great success in the field of intrusion detection. However, the high computational complexity poses a major hurdle for the practical deployment of DL-based models. In this paper, we propose a novel approach based on a lightweight deep neural network (LNN) for IDS. We design a lightweight unit that can fully extract data features while reducing the computational burden by expanding and compressing feature maps. In addition, we use inverse residual structure and channel shuffle operation to achieve more effective training. Experiment results show that our proposed model for intrusion detection not only reduces the computational cost by 61.99% and the model size by 58.84%, but also achieves satisfactory accuracy and detection rate.

Steady-state response of the grid under a predefined set of credible contingencies is an important component of power system security assessment. With the growing complexity of electrical networks, fast and reliable methods and tools are required to effectively assist transmission grid operators in making decisions concerning system security procurement. In this regard, a Convolutional Neural Network (CNN) based approach to develop prediction models for static security assessment under N-1 contingency is investigated in this paper. The CNN model is trained and applied to classify the security status of a sample system according to given node voltage magnitudes, and active and reactive power injections at network buses. Considering a set of performance metrics, the superior performance of the CNN alternative is demonstrated by comparing the obtained results with a support vector machine classifier algorithm.

Collaborative learning has gained great popularity due to its benefit of data privacy protection: participants can jointly train a Deep Learning model without sharing their training sets. However, recent works discovered that an adversary can fully recover the sensitive training samples from the shared gradients. Such reconstruction attacks pose severe threats to collaborative learning. Hence, effective mitigation solutions are urgently desired.In this paper, we propose to leverage data augmentation to defeat reconstruction attacks: by preprocessing sensitive images with carefully-selected transformation policies, it becomes infeasible for the adversary to extract any useful information from the corresponding gradients. We design a novel search method to automatically discover qualified policies. We adopt two new metrics to quantify the impacts of transformations on data privacy and model usability, which can significantly accelerate the search speed. Comprehensive evaluations demonstrate that the policies discovered by our method can defeat existing reconstruction attacks in collaborative learning, with high efficiency and negligible impact on the model performance.

Deep learning (DL) models have been shown to be vulnerable to adversarial attacks. DL model security against adversarial attacks is critical to using DL-trained models in forward deployed systems, e.g. facial recognition, document characterization, or object detection. We provide results and lessons learned applying a moving target defense (MTD) strategy against iterative, gradient-based adversarial attacks. Our strategy involves (1) training a diverse ensemble of DL models, (2) applying randomized affine input transformations to inputs, and (3) randomizing output decisions. We report a primary lesson that this strategy is ineffective against a white-box adversary, which could completely circumvent output randomization using a deterministic surrogate. We reveal how our ensemble models lacked the diversity necessary for effective MTD. We also evaluate our MTD strategy against a black-box adversary employing an ensemble surrogate model. We conclude that an MTD strategy against black-box adversarial attacks crucially depends on lack of transferability between models.

The evolution of deep learning has promoted the popularization of smart devices. However, due to the insufficient development of computing hardware, the ability to conduct local training on smart devices is greatly restricted, and it is usually necessary to deploy ready-made models. This opacity makes smart devices vulnerable to deep learning backdoor attacks. Some existing countermeasures against backdoor attacks are based on the attacker’s ignorance of defense. Once the attacker knows the defense mechanism, he can easily overturn it. In this paper, we propose a Trojaning attack defense framework based on moving target defense(MTD) strategy. According to the analysis of attack-defense game types and confrontation process, the moving target defense model based on signaling game was constructed. The simulation results show that in most cases, our technology can greatly increase the attack cost of the attacker, thereby ensuring the availability of Deep Neural Networks(DNN) and protecting it from Trojaning attacks.

We present a novel approach for the collaborative training of neural network models in decentralized federated environments. In the iterative process a group of autonomous peers run multiple training rounds to train a common model. Thereby, participants perform all model training steps locally, such as stochastic gradient descent optimization, using their private, e.g. mission-critical, training datasets. Based on locally updated models, participants can jointly determine a common model by averaging all associated model weights without sharing the actual weight values. For this purpose we introduce a simple n-out-of-n secret sharing schema and an algorithm to calculate average values in a peer-to-peer manner. Our experimental results with deep neural networks on well-known sample datasets prove the generic applicability of the approach, with regard to model quality parameters. Since there is no need to involve a central service provider in model training, the approach can help establish trustworthy collaboration platforms for businesses with high security and data protection requirements.