Biblio

Deep learning methods are increasingly becoming solutions to complex problems, including the search for anomalies. While fully-connected and convolutional neural networks have already found their application in classification problems, their applicability to the problem of detecting anomalies is limited. In this regard, it is proposed to use autoencoders, previously used only in problems of reducing the dimension and removing noise, as a method for detecting anomalies in the industrial control system. A new method based on autoencoders is proposed for detecting anomalies in the operation of industrial control systems (ICS). Several neural networks based on auto-encoders with different architectures were trained, and the effectiveness of each of them in the problem of detecting anomalies in the work of process control systems was evaluated. Auto-encoders can detect the most complex and non-linear dependencies in the data, and as a result, can show the best quality for detecting anomalies. In some cases, auto-encoders require fewer machine resources.

This paper proposes a hardware Self-Organizing Map (SOM) for high dimensional vectors. The proposed SOM is based on nested architecture with pipeline processing. Due to homogeneous modular structure, the nested architecture provides high expandability. The original nested SOM was designed to handle low-dimensional vectors with fully parallel computation, and it yielded very high performance. In this paper, the architecture is extended to handle much higher dimensional vectors by using sequential computation, which requires multiple clocks to process a single vector. To increase the performance, the proposed architecture employs pipeline computation, in which search of winner neuron and weight vector update are carried out simultaneously. Operable clock frequency for the system was 60 MHz, and its throughput reached 15012 million connection updates per second (MCUPS).

In current deep network architectures, deeper layers in networks tend to contain hundreds of independent neurons which makes it hard for humans to understand how they interact with each other. By organizing the neurons by correlation, humans can observe how clusters of neighbouring neurons interact with each other. In this paper, we propose a novel algorithm for back propagation, called Locality Guided Neural Network (LGNN) for training networks that preserves locality between neighbouring neurons within each layer of a deep network. Heavily motivated by Self-Organizing Map (SOM), the goal is to enforce a local topology on each layer of a deep network such that neighbouring neurons are highly correlated with each other. This method contributes to the domain of Explainable Artificial Intelligence (XAI), which aims to alleviate the black-box nature of current AI methods and make them understandable by humans. Our method aims to achieve XAI in deep learning without changing the structure of current models nor requiring any post processing. This paper focuses on Convolutional Neural Networks (CNNs), but can theoretically be applied to any type of deep learning architecture. In our experiments, we train various VGG and Wide ResNet (WRN) networks for image classification on CIFAR100. In depth analyses presenting both qualitative and quantitative results demonstrate that our method is capable of enforcing a topology on each layer while achieving a small increase in classification accuracy.

Malware refers to any software accessing or being installed in a system without the authorisation of administrators. Various malware has been widely used for cyber-criminals to accomplish their evil intentions and goals. To combat the increasing amount and reduce the threat of malicious programs, a novel deep learning framework, which uses NLP techniques for reference, combines CNN and LSTM neurones to capture the locally spatial correlations and learn from sequential longterm dependency is proposed. Hence, high-level abstractions and representations are automatically extracted for the malware classification task. The classification accuracy improves from 0.81 (best one by Random Forest) to approximately 1.0.

By the multi-layer nonlinear mapping and the semantic feature extraction of the deep learning, a deep learning network is proposed for video face detection to overcome the challenge of detecting faces rapidly and accurately in video with changeable background. Particularly, a pre-training procedure is used to initialize the network parameters to avoid falling into the local optimum, and the greedy layer-wise learning is introduced in the pre-training to avoid the training error transfer in layers. Key to the network is that the probability of neurons models the status of human brain neurons which is a continuous distribution from the most active to the least active and the hidden layer’s neuron number decreases layer-by-layer to reduce the redundant information of the input data. Moreover, the skin color detection is used to accelerate the detection speed by generating candidate regions. Experimental results show that, besides the faster detection speed and robustness against face rotation, the proposed method possesses lower false detection rate and lower missing detection rate than traditional algorithms.

In view of the problem that the health status and safety monitoring of the traditional intelligent home are mainly dependent on the manual inspection, this paper introduces the intelligent home-based remote monitoring system by introducing the Internet-based Internet of Things technology into the intelligent home condition monitoring and safety assessment. The system's Android remote operation based on the MVP model to develop applications, the use of neural networks to deal with users daily use of operational data to establish the network data model, combined with S3C2440A microcontrollers in the gateway to the embedded Linux to facilitate different intelligent home drivers development. Finally, the power line communication network is used to connect the intelligent electrical appliances to the gateway. By calculating the success rate of the routing nodes, the success rate of the network nodes of 15 intelligent devices is 98.33%. The system can intelligent home many electrical appliances at the same time monitoring, to solve the system data and network congestion caused by the problem can not he security monitoring.

Predicting software faults before software testing activities can help rational distribution of time and resources. Software metrics are used for software fault prediction due to their close relationship with software faults. Thanks to the non-linear fitting ability, Neural networks are increasingly used in the prediction model. We first filter metric set of the embedded software by statistical methods to reduce the dimensions of model input. Then we build a back propagation neural network with simple structure but good performance and apply it to two practical embedded software projects. The verification results show that the model has good ability to predict software faults.

Recent years, more and more testing criteria for deep learning systems has been proposed to ensure system robustness and reliability. These criteria were defined based on different perspectives of diversity. However, there lacks comprehensive investigation on what are the most essential diversities that should be considered by a testing criteria for deep learning systems. Therefore, in this paper, we conduct an empirical study to investigate the relation between test diversities and erroneous behaviors of deep learning models. We define five metrics to reflect diversities in neuron activities, and leverage metamorphic testing to detect erroneous behaviors. We investigate the correlation between metrics and erroneous behaviors. We also go further step to measure the quality of test suites under the guidance of defined metrics. Our results provided comprehensive insights on the essential diversities for testing criteria to exhibit good fault detection ability.

When we are express our internal status, such as emotions, the human body expression we use follows the compositionality principle. It is a theory in linguistic which proposes that the single components of the bodily presentation as well as the rules used to combine them are the major parts to finish this process. In this paper, such principle is applied to the process of expressing and recognizing emotional states through body expression, in which certain key features can be learned to represent certain primitives of the internal emotional state in the form of basic variables. This is done by a hierarchical recurrent neural learning framework (RNN) because of its nonlinear dynamic bifurcation, so that variables can be learned to represent different hierarchies. In addition, we applied some adaptive learning techniques in machine learning for the requirement of real-time emotion recognition, in which a stable representation can be maintained compared to previous work. The model is examined by comparing the PB values between the training and recognition phases. This hierarchical model shows the rationality of the compositionality hypothesis by the RNN learning and explains how key features can be used and combined in bodily expression to show the emotional state.

Modern infrastructure is heavily reliant on systems with interconnected computational and physical resources, named Cyber-Physical Systems (CPSs). Hence, building resilient CPSs is a prime need and continuous monitoring of the CPS operational health is essential for improving resilience. This paper presents a framework for calculating and monitoring of health in CPSs using data driven techniques. The main advantages of this data driven methodology is that the ability of leveraging heterogeneous data streams that are available from the CPSs and the ability of performing the monitoring with minimal a priori domain knowledge. The main objective of the framework is to warn the operators of any degradation in cyber, physical or overall health of the CPS. The framework consists of four components: 1) Data acquisition and feature extraction, 2) state identification and real time state estimation, 3) cyber-physical health calculation and 4) operator warning generation. Further, this paper presents an initial implementation of the first three phases of the framework on a CPS testbed involving a Microgrid simulation and a cyber-network which connects the grid with its controller. The feature extraction method and the use of unsupervised learning algorithms are discussed. Experimental results are presented for the first two phases and the results showed that the data reflected different operating states and visualization techniques can be used to extract the relationships in data features.

As an emerging paradigm for energy-efficiency design, approximate computing can reduce power consumption through simplification of logic circuits. Although calculation errors are caused by approximate computing, their impacts on the final results can be negligible in some error resilient applications, such as Convolutional Neural Networks (CNNs). Therefore, approximate computing has been applied to CNNs to reduce the high demand for computing resources and energy. Compared with the traditional method such as reducing data precision, this paper investigates the effect of approximate computing on the accuracy and power consumption of CNNs. To optimize the approximate computing technology applied to CNNs, we propose a method for quantifying the error resilience of each neuron by theoretical analysis and observe that error resilience varies widely across different neurons. On the basic of quantitative error resilience, dynamic adaptation of approximate bit-width and the corresponding configurable adder are proposed to fully exploit the error resilience of CNNs. Experimental results show that the proposed method further improves the performance of power consumption while maintaining high accuracy. By adopting the optimal approximate bit-width for each layer found by our proposed algorithm, dynamic adaptation of approximate bit-width reduces power consumption by more than 30% and causes less than 1% loss of the accuracy for LeNet-5.

To leverage the feature extraction of DBN and the fast classification and good generalization of ELM, an improved method of DBN-ELM is proposed for intrusion detection. The improved model uses deep belief network (DBN) to train NSL-KDD dataset and feed them back to the extreme learning machine (ELM) for classification. A classifier is connected at each intermediate level of the DBN-ELM. By majority voting on the output of classifier and ELM, the final output is calculated by integration. Experiments show that the improved model increases the classification confidence and accuracy of the classifier. The model has been benchmarked on the NSL-KDD dataset, and the accuracy of the model has been improved to 97.82%, while the false alarm rate has been reduced to 1.81%. Proposed improved model has been also compared with DBN, ELM, DBN-ELM and achieves competitive accuracy.

This paper mainly focuses on presenting a novel attack detection system to thread out the risk issues in IoT. The presented attack detection system links the interconnection of DevOps as it creates the correlation between development and IT operations. Further, the presented attack detection model ensures the operational security of different applications. In view of this, the implemented system incorporates two main stages named Proposed Feature Extraction process and Classification. The data from every application is processed with the initial stage of feature extraction, which concatenates the statistical and higher-order statistical features. After that, these extracted features are supplied to classification process, where determines the presence of attacks. For this classification purpose, this paper aims to deploy the optimized Deep Belief Network (DBN), where the activation function is tuned optimally. Furthermore, the optimal tuning is done by a renowned meta-heuristic algorithm called Lion Algorithm (LA). Finally, the performance of proposed work is compared and proved over other conventional methods.

Deep neural networks (DNNs) provide good performance for image recognition, speech recognition, and pattern recognition. However, a poisoning attack is a serious threat to DNN's security. The poisoning attack is a method to reduce the accuracy of DNN by adding malicious training data during DNN training process. In some situations such as a military, it may be necessary to drop only a chosen class of accuracy in the model. For example, if an attacker does not allow only nuclear facilities to be selectively recognized, it may be necessary to intentionally prevent UAV from correctly recognizing nuclear-related facilities. In this paper, we propose a selective poisoning attack that reduces the accuracy of only chosen class in the model. The proposed method reduces the accuracy of a chosen class in the model by training malicious training data corresponding to a chosen class, while maintaining the accuracy of the remaining classes. For experiment, we used tensorflow as a machine learning library and MNIST and CIFAR10 as datasets. Experimental results show that the proposed method can reduce the accuracy of the chosen class to 43.2% and 55.3% in MNIST and CIFAR10, while maintaining the accuracy of the remaining classes.

The paper reviews a project on the automation of term system construction. TSBuilder (Term System Builder) was developed in 2014 as a multilayer Rosenblatt's perceptron for supervised machine learning, namely 1-3 word terms identification in natural language texts and their rigid categorization. The program is being modified to reduce the rigidity of categorization which will bring text mining more in line with human thinking.We are expanding the range of parameters (semantical, morphological, and syntactical) for categorization, removing the restriction of the term length of three words, using convolution on a continuous sequence of terms, and present the probabilities of a term falling into different categories. The neural network will not assign a single category to a term but give N answers (where N is the number of predefined classes), each of which O ∈ [0, 1] is the probability of the term to belong to a given class.

The need for performing deep neural network inferences on resource-constrained embedded devices (e.g., Internet of Things nodes) requires specialized architectures to achieve the best trade-off among performance, energy, and cost. One of the most promising architectures in this context is based on massive parallel and specialized cores interconnected by means of a Network-on-Chip (NoC). In this paper, we extensively evaluate NoC-based deep neural network accelerators by exploring the design space spanned by several architectural parameters including, network size, routing algorithm, local memory size, link width, and number of memory interfaces. We show how latency is mainly dominated by the on-chip communication whereas energy consumption is mainly accounted by memory (both on-chip and off-chip). The outcome of the analysis, thus, pushes toward a research line devoted to the optimization of the on-chip communication fabric and the memory subsystem for performance improvement and energy efficiency, respectively.

With the continuous development of computer network technology, security problems in the network are emerging one after another, and it is becoming more and more difficult to ignore. For the current network administrators, how to successfully prevent malicious network hackers from invading, so that network systems and computers are at Safe and normal operation is an urgent task. This paper proposes a network intrusion detection method based on deep learning. This method uses deep confidence neural network to extract features of network monitoring data, and uses BP neural network as top level classifier to classify intrusion types. The method was validated using the KDD CUP'99 dataset from the Lincoln Laboratory of the Massachusetts Institute of Technology. The results show that the proposed method has a significant improvement over the traditional machine learning accuracy.

Error correction in quantum cryptography based on artificial neural networks is a new and promising solution. In this paper the security verification of this method is discussed and results of many simulations with different parameters are presented. The test scenarios assumed partially synchronized neural networks, typical for error rates in quantum cryptography. The results were also compared with scenarios based on the neural networks with random chosen weights to show the difficulty of passive attacks.

Research on network security situation awareness is currently a research hotspot in the field of network security. It is one of the easiest and most effective methods to use the BP neural network for security situation prediction. However, there are still some problems in BP neural network, such as slow convergence rate, easy to fall into local extremum, etc. On the other hand, some common used evolutionary algorithms, such as genetic algorithm (GA) and particle swarm optimization (PSO), easily fall into local optimum. Hybrid rice optimization algorithm is a newly proposed algorithm with strong search ability, so the method of this paper is proposed. This article describes in detail the use of BP network security posture prediction method. In the proposed method, HRO is used to train the connection weights of the BP network. Through the advantages of HRO global search and fast convergence, the future security situation of the network is predicted, and the accuracy of the situation prediction is effectively improved.

Current testing for Deep Neural Networks (DNNs) focuses on quantity of test cases but ignores diversity. To the best of our knowledge, DeepXplore is the first white-box framework for Deep Learning testing by triggering differential behaviors between multiple DNNs and increasing neuron coverage to improve diversity. Since it is based on multiple DNNs facing problems that (1) the framework is not friendly to a single DNN, (2) if incorrect predictions made by all DNNs simultaneously, DeepXplore cannot generate test cases. This paper presents Test4Deep, a white-box testing framework based on a single DNN. Test4Deep avoids mistakes of multiple DNNs by inducing inconsistencies between predicted labels of original inputs and that of generated test inputs. Meanwhile, Test4Deep improves neuron coverage to capture more diversity by attempting to activate more inactivated neurons. The proposed method was evaluated on three popular datasets with nine DNNs. Compared to DeepXplore, Test4Deep produced average 4.59% (maximum 10.49%) more test cases that all found errors and faults of DNNs. These test cases got 19.57% more diversity increment and 25.88% increment of neuron coverage. Test4Deep can further be used to improve the accuracy of DNNs by average up to 5.72% (maximum 7.0%).

Neural Network is used in many applications and guarding its performance against faults is a research challenge. Self-healing neural network is a promising concept for achieving reliability, which is the ability to detect and fix a fault in the system automatically. Most of the current self-healing neural network are based on replication of hardware nodes which causes significant area overhead. The proposed self-healing approach results in a modest area overhead and it is suitable for complex neural network. The proposed method is based on a shared operation and a spare node in each layer which compensates for any faulty node in the layer. Each faulty node will be compensated by its neighbor node, and the neighbor node performs the faulty node as well as its own operations sequentially. In the case the neighbor is faulty, the spare node will compensate for it. The proposed method is implemented using VHDL and the simulation results are obtained using Altira 10 GX FPGA for a different number of nodes. The area overhead is very small for a complex network. The reliability of the proposed method is studied and compared with the traditional neural network.

Two systems pursuing their own goals in a shared world can interact in ways that are not so explicit - such that the presence of another system alone can interfere with how one is able to achieve its own goals. Drawing inspiration from human psychology and the theory of social action, we propose the notion of employing social action in socially situated agents as a means of alleviating interference in interacting systems. Here we demonstrate that these specific issues of behavioural and evolutionary instability caused by the unintended consequences of interactions can be addressed with agents capable of a fusion of goal-rationality and traditional action, resulting in a stable society capable of achieving goals during the course of evolution.

Spams are unsolicited and unnecessary messages which may contain harmful codes or links for activation of malicious viruses and spywares. Increasing popularity of social networks attracts the spammers to perform malicious activities in social networks. So an efficient spam detection method is necessary for social networks. In this paper, feed forward neural network with back propagation based spam detection model is proposed. The quality of the learning process is improved by tuning initial weights of feed forward neural network using proposed enhanced step size firefly algorithm which reduces the time for finding optimal weights during the learning process. The model is applied for twitter dataset and the experimental results show that, the proposed model performs well in terms of accuracy and detection rate and has lower false positive rate. 

Recent advances in networking and communication technologies have enabled Internet-of-Things (IoT) devices to communicate more frequently and faster. An IoT device typically transmits data over the Internet which is an insecure channel. Cyber attacks such as denial-of-service (DoS), man-in-middle, and SQL injection are considered as big threats to IoT devices. In this paper, an anomaly-based intrusion detection scheme is proposed that can protect sensitive information and detect novel cyber-attacks. The Artificial Bee Colony (ABC) algorithm is used to train the Random Neural Network (RNN) based system (RNN-ABC). The proposed scheme is trained on NSL-KDD Train+ and tested for unseen data. The experimental results suggest that swarm intelligence and RNN successfully classify novel attacks with an accuracy of 91.65%. Additionally, the performance of the proposed scheme is also compared with a hybrid multilayer perceptron (MLP) based intrusion detection system using sensitivity, mean of mean squared error (MMSE), the standard deviation of MSE (SDMSE), best mean squared error (BMSE) and worst mean squared error (WMSE) parameters. All experimental tests confirm the robustness and high accuracy of the proposed scheme.

Network security became a viral topic nowadays, Anomaly-based Intrusion Detection Systems [1] (IDSs) plays an indispensable role in identifying the attacks from networks and the detection rate and accuracy are said to be high. The proposed work explore this topic and solve this issue by the IDS model developed using Artificial Neural Network (ANN). This model uses Feed - Forward Neural Net algorithms and Probabilistic Neural Network and oppositional based on Particle Swarm optimization Algorithm for lessen the computational overhead and boost the performance level. The whole computing overhead produced in its execution and training are get minimized by the various optimization techniques used in these developed ANN-based IDS system. The experimental study on the developed system tested using the standard NSL-KDD dataset performs well, while compare with other intrusion detection models, built using NN, RB and OPSO algorithms.