Biblio

In this decade, digital transactions have risen exponentially demanding more reliable and secure authentication systems. CAPTCHA (Completely Automated Public Turing Test to tell Computers and Humans Apart) system plays a major role in these systems. These CAPTCHAs are available in character sequence, picture-based, and audio-based formats. It is very essential that these CAPTCHAs should be able to differentiate a computer program from a human precisely. This work tests the strength of text-based CAPTCHAs by breaking them using an algorithm built on CNN (Convolution Neural Network) and RNN (Recurrent Neural Network). The algorithm is designed in such a way as an attempt to break the security features designers have included in the CAPTCHAs to make them hard to be cracked by machines. This algorithm is tested against the synthetic dataset generated in accordance with the schemes used in popular websites. The experiment results exhibit that the model has shown a considerable performance against both the synthetic and real-world CAPTCHAs.

In power systems, having accurate device models is crucial for grid reliability, availability, and resiliency. Existing model calibration methods based on mathematical approaches often lead to multiple solutions due to the ill-posed nature of the problem, which would require further interventions from the field engineers in order to select the optimal solution. In this paper, we present a novel deep-learning-based approach for model parameter calibration in power systems. Our study focused on the generator model as an example. We studied several deep-learning-based approaches including 1-D Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Units (GRU), which were trained to estimate model parameters using simulated Phasor Measurement Unit (PMU) data. Quantitative evaluations showed that our proposed methods can achieve high accuracy in estimating the model parameters, i.e., achieved a 0.0079 MSE on the testing dataset. We consider these promising results to be the basis for further exploration and development of advanced tools for model validation and calibration.

Intrusion Detection System (IDS) is used to identify malicious traffic on the network. Apart from rule-based IDS, machine learning and deep learning based on IDS are also being developed to improve the accuracy of IDS detection. In this study, the public dataset CIC IDS 2017 was used in developing deep learning-based IDS because this dataset contains the new types of attacks. In addition, this dataset also meets the criteria as an intrusion detection dataset. The dataset was split into train data, validation data and test data. We proposed Bidirectional Long-Short Term Memory (LSTM) for building neural network. We created 24 scenarios with various changes in training parameters which were trained for 100 epochs. The training parameters used as research variables are optimizer, activation function, and learning rate. As addition, Dropout layer and L2-regularizer were implemented on every scenario. The result shows that the model used Adam optimizer, Tanh activation function and a learning rate of 0.0001 produced the highest accuracy compared to other scenarios. The accuracy and F1 score reached 97.7264% and 97.7516%. The best model was trained again until 1000 iterations and the performance increased to 98.3448% in accuracy and 98.3793% in F1 score. The result exceeded several previous works on the same dataset.

In today's world, there is rapid progress in the field of artificial intelligence and image captioning. It becomes a fascinating task that has saw widespread interest. The task of image captioning comprises image description engendered based on the hybrid combination of deep learning, natural language processing, and various approaches of machine learning and computer vision. In this work authors emphasize on how the model generates a short description as an output of the input image using the functionalities of Deep Learning and Natural Language Processing, for helping visually impaired people, and can also be cast-off in various web sites to automate the generation of captions reducing the task of recitation with great ease.

In this fast-running world, machine communication plays a vital role. To compete with this world, human-machine interaction is a necessary thing. To enhance this, Natural Language Processing technique is used widely. Using this technique, we can reduce the interaction gap between the machine and human. Till now, many such applications are developed which are using this technique.This tool deals with the various methods which are used for development of grammar error correction. These methods include rule-based method, classifier-based method and machine translation-based method. Also, models regarding the Natural Language Processing (NLP) pipeline are trained and implemented in this project accordingly. Additionally, the tool can also perform speech to text operation.

Malicious software (malware) is designed to cause unwanted or destructive effects on computers. Since modern society is dependent on computers to function, malware has the potential to do untold damage. Therefore, developing techniques to effectively combat malware is critical. With the rise in popularity of polymorphic malware, conventional anti-malware techniques fail to keep up with the rate of emergence of new malware. This poses a major challenge towards developing an efficient and robust malware detection technique. One approach to overcoming this challenge is to classify new malware among families of known malware. Several machine learning methods have been proposed for solving the malware classification problem. However, these techniques rely on hand-engineered features extracted from malware data which may not be effective for classifying new malware. Deep learning models have shown paramount success for solving various classification tasks such as image and text classification. Recent deep learning techniques are capable of extracting features directly from the input data. Consequently, this paper proposes an end-to-end deep learning framework for multimodels (henceforth, multimodel learning) to solve the challenging malware classification problem. The proposed model utilizes three different deep neural network architectures to jointly learn meaningful features from different attributes of the malware data. End-to-end learning optimizes all processing steps simultaneously, which improves model accuracy and generalizability. The performance of the model is tested with the widely used and publicly available Microsoft Malware Challenge Dataset and is compared with the state-of-the-art deep learning-based malware classification pipeline. Our results suggest that the proposed model achieves comparable performance to the state-of-the-art methods while offering faster training using end-to-end multimodel learning.

The paper proposes a novel technique of EEG induced Brain-Computer Interface system for user authentication of personal devices. The scheme enables a human user to lock and unlock any personal device using his/her mind generated password. A two stage security verification is employed in the scheme. In the first stage, a 3 × 3 spatial matrix of flickering circles will appear on the screen of which, rows are blinked randomly and user has to mentally select a row which contains his desired circle.P300 is released when the desired row is blinked. Successful selection of row is followed by the selection of a flickering circle in the desired row. Gazing at a particular flickering circle generates SSVEP brain pattern which is decoded to trace the mentally selected circle. User is able to store mentally uttered number in the selected circle, later the number with it's spatial position will serve as the password for the unlocking phase. Here, the user is equipped with a headphone where numbers starting from zero to nine are spelled randomly. Spelled number matching with the mentally uttered number generates auditory P300 in the subject's brain. The particular choice of mentally uttered number is detected by successful detection of auditory P300. A novel weight update algorithm of Recurrent Neural Network (RNN), based on Extended-Kalman Filter and Particle Filter is used here for classifying the brain pattern. The proposed classifier achieves the best classification accuracy of 95.6%, 86.5% and 83.5% for SSVEP, visual P300 and auditory P300 respectively.

Artificial neural networks allow the identification of black-box models. This paper proposes a method aimed at replicating the static and dynamic behavior of a DC-DC power converter based on a recurrent nonlinear autoregressive exogenous neural network. The method proposed in this work applies an algorithm that trains a neural network based on the inputs and outputs (currents and voltages) of a Buck converter. The approach is validated by means of simulated data of a realistic nonsynchronous Buck converter model programmed in Simulink and by means of experimental results. The predictions made by the neural network are compared to the actual outputs of the system, to determine the accuracy of the method, thus validating the proposed approach. Both simulation and experimental results show the feasibility and accuracy of the proposed black-box approach.

Inferring behavior model of a running software system is quite useful for several automated software engineering tasks, such as program comprehension, anomaly detection, and testing. Most existing dynamic model inference techniques are white-box, i.e., they require source code to be instrumented to get run-time traces. However, in many systems, instrumenting the entire source code is not possible (e.g., when using black-box third-party libraries) or might be very costly. Unfortunately, most black-box techniques that detect states over time are either univariate, or make assumptions on the data distribution, or have limited power for learning over a long period of past behavior. To overcome the above issues, in this paper, we propose a hybrid deep neural network that accepts as input a set of time series, one per input/output signal of the system, and applies a set of convolutional and recurrent layers to learn the non-linear correlations between signals and the patterns, over time. We have applied our approach on a real UAV auto-pilot solution from our industry partner with half a million lines of C code. We ran 888 random recent system-level test cases and inferred states, over time. Our comparison with several traditional time series change point detection techniques showed that our approach improves their performance by up to 102%, in terms of finding state change points, measured by F1 score. We also showed that our state classification algorithm provides on average 90.45% F1 score, which improves traditional classification algorithms by up to 17%.

In recent years, there has been a significant increase in network intrusion attacks which raises a great concern from the privacy and security aspects. Due to the advancement of the technology, cyber-security attacks are becoming very complex such that the current detection systems are not sufficient enough to address this issue. Therefore, an implementation of an intelligent and effective network intrusion detection system would be crucial to solve this problem. In this paper, we use deep learning techniques, namely, Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) to design an intelligent detection system which is able to detect different network intrusions. Additionally, we evaluate the performance of the proposed solution using different evaluation matrices and we present a comparison between the results of our proposed solution to find the best model for the network intrusion detection system.

We investigate a deep learning model for action recognition that simultaneously extracts spatio-temporal information from a raw RGB input data. The proposed multiple spatio-temporal scales recurrent neural network (MSTRNN) model is derived by combining multiple timescale recurrent dynamics with a conventional convolutional neural network model. The architecture of the proposed model imposes both spatial and temporal constraints simultaneously on its neural activities. The constraints vary, with multiple scales in different layers. As suggested by the principle of upward and downward causation, it is assumed that the network can develop a functional hierarchy using its constraints during training. To evaluate and observe the characteristics of the proposed model, we use three human action datasets consisting of different primitive actions and different compositionality levels. The performance capabilities of the MSTRNN model on these datasets are compared with those of other representative deep learning models used in the field. The results show that the MSTRNN outperforms baseline models while using fewer parameters. The characteristics of the proposed model are observed by analyzing its internal representation properties. The analysis clarifies how the spatio-temporal constraints of the MSTRNN model aid in how it extracts critical spatio-temporal information relevant to its given tasks.

In recent years, unmanned aerial vehicles (UAVs) have become increasingly accessible to the public due to their high availability with affordable prices while being equipped with better technology. However, this raises a great concern from both the cyber and physical security perspectives since UAVs can be utilized for malicious activities in order to exploit vulnerabilities by spying on private properties, critical areas or to carry dangerous objects such as explosives which makes them a great threat to the society. Drone identification is considered the first step in a multi-procedural process in securing physical infrastructure against this threat. In this paper, we present drone detection and identification methods using deep learning techniques such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Convolutional Recurrent Neural Network (CRNN). These algorithms will be utilized to exploit the unique acoustic fingerprints of the flying drones in order to detect and identify them. We propose a comparison between the performance of different neural networks based on our dataset which features audio recorded samples of drone activities. The major contribution of our work is to validate the usage of these methodologies of drone detection and identification in real life scenarios and to provide a robust comparison of the performance between different deep neural network algorithms for this application. In addition, we are releasing the dataset of drone audio clips for the research community for further analysis.

In the age of knowledge, Natural Language Processing (NLP) express its demand by a huge range of utilization. Previously NLP was dealing with statically data. Contemporary time NLP is doing considerably with the corpus, lexicon database, pattern reorganization. Considering Deep Learning (DL) method recognize artificial Neural Network (NN) to nonlinear process, NLP tools become increasingly accurate and efficient that begin a debacle. Multi-Layer Neural Network obtaining the importance of the NLP for its capability including standard speed and resolute output. Hierarchical designs of data operate recurring processing layers to learn and with this arrangement of DL methods manage several practices. In this paper, this resumed striving to reach a review of the tools and the necessary methodology to present a clear understanding of the association of NLP and DL for truly understand in the training. Efficiency and execution both are improved in NLP by Part of speech tagging (POST), Morphological Analysis, Named Entity Recognition (NER), Semantic Role Labeling (SRL), Syntactic Parsing, and Coreference resolution. Artificial Neural Networks (ANN), Time Delay Neural Networks (TDNN), Recurrent Neural Network (RNN), Convolution Neural Networks (CNN), and Long-Short-Term-Memory (LSTM) dealings among Dense Vector (DV), Windows Approach (WA), and Multitask learning (MTL) as a characteristic of Deep Learning. After statically methods, when DL communicate the influence of NLP, the individual form of the NLP process and DL rule collaboration was started a fundamental connection.

In recent years, the spreading of malicious social media messages about financial stocks has threatened the security of financial market. Market Anomaly Attacks is an illegal practice in the stock or commodities markets that induces investors to make purchase or sale decisions based on false information. Identifying these threats from noisy social media datasets remains challenging because of the long time sequence in these social media postings, ambiguous textual context and the difficulties for traditional deep learning approaches to handle both temporal and text dependent data such as financial social media messages. This research developed a temporal recurrent neural network (TRNN) approach to capturing both time and text sequence dependencies for intelligent detection of market anomalies. We tested the approach by using financial social media of U.S. technology companies and their stock returns. Compared with traditional neural network approaches, TRNN was found to more efficiently and effectively classify abnormal returns.

Network attack is a significant security issue for modern society. From small mobile devices to large cloud platforms, almost all computing products, used in our daily life, are networked and potentially under the threat of network intrusion. With the fast-growing network users, network intrusions become more and more frequent, volatile and advanced. Being able to capture intrusions in time for such a large scale network is critical and very challenging. To this end, the machine learning (or AI) based network intrusion detection (NID), due to its intelligent capability, has drawn increasing attention in recent years. Compared to the traditional signature-based approaches, the AI-based solutions are more capable of detecting variants of advanced network attacks. However, the high detection rate achieved by the existing designs is usually accompanied by a high rate of false alarms, which may significantly discount the overall effectiveness of the intrusion detection system. In this paper, we consider the existence of spatial and temporal features in the network traffic data and propose a hierarchical CNN+RNN neural network, LuNet. In LuNet, the convolutional neural network (CNN) and the recurrent neural network (RNN) learn input traffic data in sync with a gradually increasing granularity such that both spatial and temporal features of the data can be effectively extracted. Our experiments on two network traffic datasets show that compared to the state-of-the-art network intrusion detection techniques, LuNet not only offers a high level of detection capability but also has a much low rate of false positive-alarm.

With the increasing diversity of Android malware, the effectiveness of conventional defense mechanisms are at risk. This situation has endorsed a notable interest in the improvement of the exactitude and scalability of malware detection for smart devices. In this study, we have proposed an effective deep learning-based malware detection model for competent and improved ransomware detection in Android environment by looking at the algorithm of Long Short-Term Memory (LSTM). The feature selection has been done using 8 different feature selection algorithms. The 19 important features are selected through simple majority voting process by comparing results of all feature filtration techniques. The proposed algorithm is evaluated using android malware dataset (CI-CAndMal2017) and standard performance parameters. The proposed model outperforms with 97.08% detection accuracy. Based on outstanding performance, we endorse our proposed algorithm to be efficient in malware and forensic analysis.

The Internet has gradually penetrated into the national economy, politics, culture, military, education and other fields. Due to its openness, interconnectivity and other characteristics, the Internet is vulnerable to all kinds of malicious attacks. The research uses a honeynet to collect attacker information, and proposes a network penetration recognition technology based on interactive behavior analysis. Using Sebek technology to capture the attacker's keystroke record, time series modeling of the keystroke sequences of the interaction behavior is proposed, using a Recurrent Neural Network. The attack recognition method is constructed by using Long Short-Term Memory that solves the problem of gradient disappearance, gradient explosion and long-term memory shortage in ordinary Recurrent Neural Network. Finally, the experiment verifies that the short-short time memory network has a high accuracy rate for the recognition of penetration attacks.

Conversational agents have very much become part of our lives since the renaissance of neural network based "neural conversational agents". Previously used manually annotated and rule based methods lacked the scalability and generalization capabilities of the neural conversational agents. A neural conversational agent has two parts: at one end an encoder understands the question while the other end a decoder prepares and outputs the corresponding answer to the question asked. Both the parts are typically designed using recurrent neural network and its variants and trained in an end-to-end fashion. Although conversation agents for other languages have been developed, Urdu language has seen very less progress in building of conversational agents. Especially recent state of the art neural network based techniques have not been explored yet. In this paper, we design an attention driven deep encoder-decoder based neural conversational agent for Urdu language. Overall, we make following contributions we (i) create a dataset of 5000 question-answer pairs, and (ii) present a new deep encoder-decoder based conversational agent for Urdu language. For our work, we limit the knowledge base of our agent to general knowledge regarding Pakistan. Our best model has the BLEU score of 58 and gives syntactically and semantically correct answers in majority of the cases.

As the era of cloud technology arises, more and more people are beginning to migrate their applications and personal data to the cloud. This makes web-based applications an attractive target for cyber-attacks. As a result, web-based applications now need more protections than ever. However, current anomaly-based web attack detection approaches face the difficulties like unsatisfying accuracy and lack of generalization. And the rule-based web attack detection can hardly fight unknown attacks and is relatively easy to bypass. Therefore, we propose a novel deep learning approach to detect anomalous requests. Our approach is to first train two Recurrent Neural Networks (RNNs) with the complicated recurrent unit (LSTM unit or GRU unit) to learn the normal request patterns using only normal requests unsupervisedly and then supervisedly train a neural network classifier which takes the output of RNNs as the input to discriminate between anomalous and normal requests. We tested our model on two datasets and the results showed that our model was competitive with the state-of-the-art. Our approach frees us from feature selection. Also to the best of our knowledge, this is the first time that the RNN is applied on anomaly-based web attack detection systems.

Recent methods for learning vector space representations of words, word embedding, such as GloVe and Word2Vec have succeeded in capturing fine-grained semantic and syntactic regularities. We analyzed the effectiveness of these methods for e-commerce recommender systems by transferring the sequence of items generated by users' browsing journey in an e-commerce website into a sentence of words. We examined the prediction of fine-grained item similarity (such as item most similar to iPhone 6 64GB smart phone) and item analogy (such as iPhone 5 is to iPhone 6 as Samsung S5 is to Samsung S6) using real life users' browsing history of an online European department store. Our results reveal that such methods outperform related models such as singular value decomposition (SVD) with respect to item similarity and analogy tasks across different product categories. Furthermore, these methods produce a highly condensed item vector space representation, item embedding, with behavioral meaning sub-structure. These vectors can be used as features in a variety of recommender system applications. In particular, we used these vectors as features in a neural network based models for anonymous user recommendation based on session's first few clicks. It is found that recurrent neural network that preserves the order of user's clicks outperforms standard neural network, item-to-item similarity and SVD (recall@10 value of 42% based on first three clicks) for this task.

Preparing recommendations for unknown users or such that correctly respond to the short-term needs of a particular user is one of the fundamental problems for e-commerce. Most of the common Recommender Systems assume that user identification must be explicit. In this paper a Session-Aware Recommender System approach is presented where no straightforward user information is required. The recommendation process is based only on user activity within a single session, defined as a sequence of events. This information is incorporated in the recommendation process by explicit context modeling with factorization methods and a novel approach with Recurrent Neural Network (RNN). Compared to the session modeling approach, RNN directly models the dependency of user observed sequential behavior throughout its recurrent structure. The evaluation discusses the results based on sessions from real-life system with ephemeral items (identified only by the set of their attributes) for the task of top-n best recommendations.

Tracking moving objects is a task of the utmost importance to the defence community. As this task requires high accuracy, rather than employing a single detector, it has become common to use multiple ones. In such cases, the tracks produced by these detectors need to be correlated (if they belong to the same sensing modality) or associated (if they were produced by different sensing modalities). In this work, we introduce Computational-Intelligence-based methods for correlating and associating various contacts and tracks pertaining to maritime vessels in an area of interest. Fuzzy k-Nearest Neighbours will be used to conduct track correlation and Fuzzy C-Means clustering will be applied for association. In that way, the uncertainty of the track correlation and association is handled through fuzzy logic. To better model the state of the moving target, the traditional Kalman Filter will be extended using an Echo State Network. Experimental results on five different types of sensing systems will be discussed to justify the choices made in the development of our approach. In particular, we will demonstrate the judiciousness of using Fuzzy k-Nearest Neighbours and Fuzzy C-Means on our tracking system and show how the extension of the traditional Kalman Filter by a recurrent neural network is superior to its extension by other methods.