Biblio

Robustness verification of neural networks (NNs) is a challenging and significant problem, which draws great attention in recent years. Existing researches have shown that bound propagation is a scalable and effective method for robustness verification, and it can be implemented on GPUs and TPUs to get parallelized. However, the bound propagation methods naturally produce weak bound due to linear relaxations on the neurons, which may cause failure in verification. Although tightening techniques for simple ReLU networks have been explored, they are not applicable for NNs with general activation functions such as Sigmoid and Tanh. Improving robustness verification on these NNs is still challenging. In this paper, we propose a Branch-and-Bound (BaB) style method to address this problem. The proposed BaB procedure improves the weak bound by splitting the input domain of neurons into sub-domains and solving the corresponding sub-problems. We propose a generic heuristic function to determine the priority of neuron splitting by scoring the relaxation and impact of neurons. Moreover, we combine bound optimization with the BaB procedure to improve the weak bound. Experimental results demonstrate that the proposed method gains up to 35% improvement compared to the state-of-art CROWN method on Sigmoid and Tanh networks.

An ultra-low-power gesture and gait classification SoC is presented for rehabilitation application featuring (1) mixed-signal feature extraction and integrated low-noise amplifier eliminating expensive ADC and digital feature extraction, (2) an integrated distributed deep neural network (DNN) ASIC supporting a scalable multi-chip neural network for sensor fusion with distortion resiliency for low-cost front end modules, (3) onchip learning of DNN engine allowing in-situ training of user specific operations. A 12-channel 65nm CMOS test chip was fabricated with 1μW power per channel, less than 3ms computation latency, on-chip training for user-specific DNN model and multi-chip networking capability.

The results of evaluating explanations of the black-box model for prediction are presented. The XAI evaluation is realized through the different principles and characteristics between black-box model explanations and XAI labels. In the field of high-dimensional prediction, the black-box model represented by neural network and ensemble models can predict complex data sets more accurately than traditional linear regression and white-box models such as the decision tree model. However, an unexplainable characteristic not only hinders developers from debugging but also causes users mistrust. In the XAI field dedicated to ``opening'' the black box model, effective evaluation methods are still being developed. Within the established XAI evaluation framework (MDMC) in this paper, explanation methods for the prediction can be effectively tested, and the identified explanation method with relatively higher quality can improve the accuracy, transparency, and reliability of prediction.

Deep learning is becoming a basis of decision making systems in many application domains, such as autonomous vehicles, health systems, etc., where the risk of misclassification can lead to serious consequences. It is necessary to know to which extent are Deep Neural Networks (DNNs) robust against various types of adversarial conditions. In this paper, we experimentally evaluate DNNs implemented in embedded device by using laser fault injection, a physical attack technique that is mostly used in security and reliability communities to test robustness of various systems. We show practical results on four activation functions, ReLu, softmax, sigmoid, and tanh. Our results point out the misclassification possibilities for DNNs achieved by injecting faults into the hidden layers of the network. We evaluate DNNs by using several different attack strategies to show which are the most efficient in terms of misclassification success rates. Outcomes of this work should be taken into account when deploying devices running DNNs in environments where malicious attacker could tamper with the environmental parameters that would bring the device into unstable conditions. resulting into faults.

Internet of Things (IoT) development and the addition of billions of computationally limited devices prohibit the use of classical security measures such as Intrusion Detection Systems (IDS). In this paper, we study the influence of the implementation of different feed-forward type of Neural Networks (NNs) on the detection Rate of the Negative Selection Neural Network (NSNN) algorithm. Feed-forward and cascade forward NN structures with different number of neurons and different number of hidden layers are tested. For training and testing the NSNN algorithm the labeled KDD NSL dataset is applied. The detection rates provided by the algorithm with several NN structures to determine the optimal solution are calculated and compared. The results show how these different feed-forward based NN architectures impact the performance of the NSNN algorithm.

Cyber-security is facing a huge threat from malware and malware mass production due to its mutation factors. Classification of malware by their features is necessary for the security of information technology (IT) society. To provide security from malware, deep neural networks (DNN) can offer a superior solution for the detection and categorization of malware samples by using image classification techniques. To strengthen our ideology of malware classification through image recognition, we have experimented by comparing two perspectives of malware classification. The first perspective implements dense neural networks on binary files and the other applies deep layered convolutional neural network on malware images. The proposed model is trained to a set of malware samples, which are further distributed into 9 different families. The dataset of malware samples which is used in this paper is provided by Microsoft for Microsoft Malware Classification Challenge in 2015. The proposed model shows an accuracy of 97.80% on the provided dataset. By using the proposed model optimum classifications results can be attained.

In this paper, an Intrusion Detection System (IDS) in the Controller Area Network (CAN) bus of modern vehicles has been proposed. NESLIDS is an anomaly detection algorithm based on the supervised Deep Neural Network (DNN) architecture that is designed to counter three critical attack categories: Denial-of-service (DoS), fuzzy, and impersonation attacks. Our research scope included modifying DNN parameters, e.g. number of hidden layer neurons, batch size, and activation functions according to how well it maximized detection accuracy and minimized the false positive rate (FPR) for these attacks. Our methodology consisted of collecting CAN Bus data from online and in real-time, injecting attack data after data collection, preprocessing in Python, training the DNN, and testing the model with different datasets. Results show that the proposed IDS effectively detects all attack types for both types of datasets. NESLIDS outperforms existing approaches in terms of accuracy, scalability, and low false alarm rates.

The existence of adversarial samples makes the security of machine learning models in practical application questioned, especially the black-box adversarial attack, which is very close to the actual application scenario. Efficient search for black-box attack samples is helpful to train more robust models. We discuss the situation that the attacker can get nothing except the final predict label. As for this problem, the current state-of-the-art method is Boundary Attack(BA) and its variants, such as Biased Boundary Attack(BBA), however it still requires large number of queries and kills a lot of time. In this paper, we propose a novel method to solve these shortcomings. First, we improved the algorithm for generating initial adversarial samples with smaller L2 distance. Second, we innovatively combine a swarm intelligence algorithm - Particle Swarm Optimization(PSO) with Biased Boundary Attack and propose PSO-BBA method. Finally, we experiment on ImageNet dataset, and compared our algorithm with the baseline algorithm. The results show that:(1)our improved initial point selection algorithm effectively reduces the number of queries;(2)compared with the most advanced methods, our PSO-BBA method improves the convergence speed while ensuring the attack accuracy;(3)our method has a good effect on both targeted attack and untargeted attack.

Nowadays, the networks attacks became very sophisticated and hard to be recognized, The traditional types of intrusion detection systems became inefficient in predicting new types of attacks. As the IDS is an important factor in securing the network in the real time, many new effective IDS approaches have been proposed. In this paper, we intend to discuss different Artificial Neural Networks based IDS approaches, also we are going to categorize them in four categories (normal ANN, DNN, CNN, RNN) and make a comparison between them depending on different performance parameters (accuracy, FNR, FPR, training time, epochs and the learning rate) and other factors like the network structure, the classification type, the used dataset. At the end of the survey, we will mention the merits and demerits of each approach and suggest some enhancements to avoid the noticed drawbacks.

Due to their proven efficiency, machine-learning systems are deployed in a wide range of complex real-life problems. More specifically, Spiking Neural Networks (SNNs) emerged as a promising solution to the accuracy, resource-utilization, and energy-efficiency challenges in machine-learning systems. While these systems are going mainstream, they have inherent security and reliability issues. In this paper, we propose NeuroAttack, a cross-layer attack that threatens the SNNs integrity by exploiting low-level reliability issues through a high-level attack. Particularly, we trigger a fault-injection based sneaky hardware backdoor through a carefully crafted adversarial input noise. Our results on Deep Neural Networks (DNNs) and SNNs show a serious integrity threat to state-of-the art machine-learning techniques.

The new switching architecture of the neural networks was proposed. The switching neural networks consist of the neurons and the switchers. The goal is to reduce expenses on the artificial neural network design and training. For realization of complex models, algorithms and methods of management the neural networks of the big size are required. The number of the interconnection links “everyone with everyone” grows with the number of neurons. The training of big neural networks requires the resources of supercomputers. Time of training of neural networks also depends on the number of neurons in the network. Switching neural networks are divided into fragments connected by the switchers. Training of switcher neuron network is provided by fragments. On the basis of switching neural networks the devices of associative memory were designed with the number of neurons comparable to the human brain.

The object of research is information security. The tools used for research are artificial neural networks. The goal is to increase the cryptography security. The problems are: the big volume of information, the expenses for neural networks design and training. It is offered to use the neural network for the cryptographic transformation of information.

As neural networks make their way into safety-critical systems, where misbehavior can lead to catastrophes, there is a growing interest in certifying the equivalence of two structurally similar neural networks - a problem known as differential verification. For example, compression techniques are often used in practice for deploying trained neural networks on computationally- and energy-constrained devices, which raises the question of how faithfully the compressed network mimics the original network. Unfortunately, existing methods either focus on verifying a single network or rely on loose approximations to prove the equivalence of two networks. Due to overly conservative approximation, differential verification lacks scalability in terms of both accuracy and computational cost. To overcome these problems, we propose NEURODIFF, a symbolic and fine-grained approximation technique that drastically increases the accuracy of differential verification on feed-forward ReLU networks while achieving many orders-of-magnitude speedup. NEURODIFF has two key contributions. The first one is new convex approximations that more accurately bound the difference of two networks under all possible inputs. The second one is judicious use of symbolic variables to represent neurons whose difference bounds have accumulated significant error. We find that these two techniques are complementary, i.e., when combined, the benefit is greater than the sum of their individual benefits. We have evaluated NEURODIFF on a variety of differential verification tasks. Our results show that NEURODIFF is up to 1000X faster and 5X more accurate than the state-of-the-art tool.

A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns new attack sequences and improves its own internal penetration testing logic. As a result, this agent (AgentPen for simplicity) is able to successfully penetrate hosts it has never interacted with before. A computer security administrator using this tool would receive a comprehensive, automated sequence of actions leading to a security breach, highlighting potential vulnerabilities, and reducing the amount of menial tasks a typical penetration tester would need to execute. To achieve autonomy, we apply an unsupervised machine learning algorithm, Q-learning, with an approximator that incorporates a deep neural network architecture. The security audit itself is modelled as a Markov Decision Process in order to test a number of decision-making strategies and compare their convergence to optimality. A series of experimental results is presented to show how this approach can be effectively used to automate penetration testing using a scalable, i.e. not exhaustive, and adaptive approach.

Spiking Neural Networks (SNNs) claim to present many advantages in terms of biological plausibility and energy efficiency compared to standard Deep Neural Networks (DNNs). Recent works have shown that DNNs are vulnerable to adversarial attacks, i.e., small perturbations added to the input data can lead to targeted or random misclassifications. In this paper, we aim at investigating the key research question: "Are SNNs secure?" Towards this, we perform a comparative study of the security vulnerabilities in SNNs and DNNs w.r.t. the adversarial noise. Afterwards, we propose a novel black-box attack methodology, i.e., without the knowledge of the internal structure of the SNN, which employs a greedy heuristic to automatically generate imperceptible and robust adversarial examples (i.e., attack images) for the given SNN. We perform an in-depth evaluation for a Spiking Deep Belief Network (SDBN) and a DNN having the same number of layers and neurons (to obtain a fair comparison), in order to study the efficiency of our methodology and to understand the differences between SNNs and DNNs w.r.t. the adversarial examples. Our work opens new avenues of research towards the robustness of the SNNs, considering their similarities to the human brain's functionality.

Hyperbox classifier is an efficient tool for modern pattern classification problems due to its transparency and rigorous use of Euclidian geometry. Fuzzy min-max (FMM) network efficiently implements the hyperbox classifier, and has been modified several times to yield better classification accuracy. However, the obtained accuracy is not up to the mark. Therefore, in this paper, a new improved FMM (IFMM) network is proposed to increase the accuracy rate. In the proposed IFMM network, a modified constraint is employed to check the expandability of a hyperbox. It also uses semiperimeter of the hyperbox along with k-nearest mechanism to select the expandable hyperbox. In the proposed IFMM, the contraction rules of conventional FMM and enhanced FMM (EFMM) are also modified using semiperimeter of a hyperbox in order to balance the size of both overlapped hyperboxes. Experimental results show that the proposed IFMM network outperforms the FMM, k-nearest FMM, and EFMM by yielding more accuracy rate with less number of hyperboxes. The proposed methods are also applied to histopathological images to know the best magnification factor for classification.

This paper showcases multiclass classification baselines using different machine learning algorithms and neural networks for distinguishing legitimate network traffic from direct and obfuscated network intrusions. This research derives its baselines from Advanced Security Network Metrics & Tunneling Obfuscations dataset. The dataset captured legitimate and obfuscated malicious TCP communications on selected vulnerable network services. The multiclass classification NIDS is able to distinguish obfuscated and direct network intrusion with up to 95% accuracy.

Data privacy is a very important problem to address while sharing data among multiple organizations and has become very crucial in the health sectors since multiple organizations such as hospitals are storing data of patients in the form of Electronic Health Records. Stored data is used with other organizations or research analysts to improve the health care of patients. However, the data records contain sensitive information such as age, sex, and date of birth of the patients. Revealing sensitive data can cause a privacy breach of the individuals. This has triggered research that has led to many different privacy preserving techniques being introduced. Thus, we designed a technique that not only encrypts / hides the sensitive information but also sends the data to different organizations securely. To encrypt sensitive data we use different fuzzy logic membership functions. We then use an autoencoder neural network to send the modified data. The output data of the autoencoder can then be used by different organizations for research analysis.

In painting, humans can draw an interrelation between the style and the content of a given image in order to enhance visual experiences. Deep neural networks like convolutional neural networks are being used to draw a satisfying conclusion of this problem of neural style transfer due to their exceptional results in the key areas of visual perceptions such as object detection and face recognition.In this study, along with style transfer on whole image it is also outlined how transfer of style can be performed only on the specific parts of the content image which is accomplished by using masks. The style is transferred in a way that there is a least amount of loss to the content image i.e., semantics of the image is preserved.

Predicting software faults before software testing activities can help rational distribution of time and resources. Software metrics are used for software fault prediction due to their close relationship with software faults. Thanks to the non-linear fitting ability, Neural networks are increasingly used in the prediction model. We first filter metric set of the embedded software by statistical methods to reduce the dimensions of model input. Then we build a back propagation neural network with simple structure but good performance and apply it to two practical embedded software projects. The verification results show that the model has good ability to predict software faults.

We investigate a deep learning model for action recognition that simultaneously extracts spatio-temporal information from a raw RGB input data. The proposed multiple spatio-temporal scales recurrent neural network (MSTRNN) model is derived by combining multiple timescale recurrent dynamics with a conventional convolutional neural network model. The architecture of the proposed model imposes both spatial and temporal constraints simultaneously on its neural activities. The constraints vary, with multiple scales in different layers. As suggested by the principle of upward and downward causation, it is assumed that the network can develop a functional hierarchy using its constraints during training. To evaluate and observe the characteristics of the proposed model, we use three human action datasets consisting of different primitive actions and different compositionality levels. The performance capabilities of the MSTRNN model on these datasets are compared with those of other representative deep learning models used in the field. The results show that the MSTRNN outperforms baseline models while using fewer parameters. The characteristics of the proposed model are observed by analyzing its internal representation properties. The analysis clarifies how the spatio-temporal constraints of the MSTRNN model aid in how it extracts critical spatio-temporal information relevant to its given tasks.

The paper reviews a project on the automation of term system construction. TSBuilder (Term System Builder) was developed in 2014 as a multilayer Rosenblatt's perceptron for supervised machine learning, namely 1-3 word terms identification in natural language texts and their rigid categorization. The program is being modified to reduce the rigidity of categorization which will bring text mining more in line with human thinking.We are expanding the range of parameters (semantical, morphological, and syntactical) for categorization, removing the restriction of the term length of three words, using convolution on a continuous sequence of terms, and present the probabilities of a term falling into different categories. The neural network will not assign a single category to a term but give N answers (where N is the number of predefined classes), each of which O ∈ [0, 1] is the probability of the term to belong to a given class.

Intrusion detection system (IDS) has become an essential layer in all the latest ICT system due to an urge towards cyber safety in the day-to-day world. Reasons including uncertainty in finding the types of attacks and increased the complexity of advanced cyber attacks, IDS calls for the need of integration of Deep Neural Networks (DNNs). In this paper, DNNs have been utilized to predict the attacks on Network Intrusion Detection System (N-IDS). A DNN with 0.1 rate of learning is applied and is run for 1000 number of epochs and KDDCup-`99' dataset has been used for training and benchmarking the network. For comparison purposes, the training is done on the same dataset with several other classical machine learning algorithms and DNN of layers ranging from 1 to 5. The results were compared and concluded that a DNN of 3 layers has superior performance over all the other classical machine learning algorithms.

In this paper we solve the problem of neural network technology development for e-mail messages classification. We analyze basic methods of spam filtering such as a sender IP-address analysis, spam messages repeats detection and the Bayesian filtering according to words. We offer the neural network technology for solving this problem because the neural networks are universal approximators and effective in addressing the problems of classification. Also, we offer the scheme of this technology for e-mail messages “spam”/“not spam” classification. The creation of effective neural network model of spam filtering is performed within the databases knowledge discovery technology. For this training set is formed, the neural network model is trained, its value and classifying ability are estimated. The experimental studies have shown that a developed artificial neural network model is adequate and it can be effectively used for the e-mail messages classification. Thus, in this paper we have shown the possibility of the effective neural network model use for the e-mail messages filtration and have shown a scheme of artificial neural network model use as a part of the e-mail spam filtering intellectual system.

Research on network security situation awareness is currently a research hotspot in the field of network security. It is one of the easiest and most effective methods to use the BP neural network for security situation prediction. However, there are still some problems in BP neural network, such as slow convergence rate, easy to fall into local extremum, etc. On the other hand, some common used evolutionary algorithms, such as genetic algorithm (GA) and particle swarm optimization (PSO), easily fall into local optimum. Hybrid rice optimization algorithm is a newly proposed algorithm with strong search ability, so the method of this paper is proposed. This article describes in detail the use of BP network security posture prediction method. In the proposed method, HRO is used to train the connection weights of the BP network. Through the advantages of HRO global search and fast convergence, the future security situation of the network is predicted, and the accuracy of the situation prediction is effectively improved.