Biblio

Recent advancements in Deep Neural Networks (DNNs) have enabled widespread deployment in multiple security-sensitive domains. The need for resource-intensive training and the use of valuable domain-specific training data have made these models the top intellectual property (IP) for model owners. One of the major threats to DNN privacy is model extraction attacks where adversaries attempt to steal sensitive information in DNN models. In this work, we propose an advanced model extraction framework DeepSteal that steals DNN weights remotely for the first time with the aid of a memory side-channel attack. Our proposed DeepSteal comprises two key stages. Firstly, we develop a new weight bit information extraction method, called HammerLeak, through adopting the rowhammer-based fault technique as the information leakage vector. HammerLeak leverages several novel system-level techniques tailored for DNN applications to enable fast and efficient weight stealing. Secondly, we propose a novel substitute model training algorithm with Mean Clustering weight penalty, which leverages the partial leaked bit information effectively and generates a substitute prototype of the target victim model. We evaluate the proposed model extraction framework on three popular image datasets (e.g., CIFAR-10/100/GTSRB) and four DNN architectures (e.g., ResNet-18/34/Wide-ResNetNGG-11). The extracted substitute model has successfully achieved more than 90% test accuracy on deep residual networks for the CIFAR-10 dataset. Moreover, our extracted substitute model could also generate effective adversarial input samples to fool the victim model. Notably, it achieves similar performance (i.e., 1-2% test accuracy under attack) as white-box adversarial input attack (e.g., PGD/Trades).

OHODIN is an online extension for data streams of the kNN-based ODIN anomaly detection approach. It provides a detection-threshold heuristic that is based on extreme value theory. In contrast to sophisticated anomaly and novelty detection approaches the decision-making process of ODIN is interpretable by humans, making it interesting for certain applications. However, it is limited in terms of the underlying detection method. In this article, we present an extension of the OHODIN to further detection techniques to reinforce OHODIN capability of online data streams anomaly detection. We introduce the algorithm modifications and an experimental evaluation with competing state-of-the-art anomaly detection approaches.

Container security has received much research attention recently. Previous work has proposed to apply various machine learning techniques to detect security attacks in containerized applications. On one hand, supervised machine learning schemes require sufficient labelled training data to achieve good attack detection accuracy. On the other hand, unsupervised machine learning methods are more practical by avoiding training data labelling requirements, but they often suffer from high false alarm rates. In this paper, we present SHIL, a self-supervised hybrid learning solution, which combines unsupervised and supervised learning methods to achieve high accuracy without requiring any manual data labelling. We have implemented a prototype of SHIL and conducted experiments over 41 real world security attacks in 28 commonly used server applications. Our experimental results show that SHIL can reduce false alarms by 39-91% compared to existing supervised or unsupervised machine learning schemes while achieving a higher or similar detection rate.

Phishing is a type of fraud on the Internet in the form of fake web pages that mimic the original web pages to trick users into sending sensitive information to phisher. The statistics presented by APWG and Phistank show that the number of phishing websites from 2015 to 2020 tends to increase continuously. To overcome this problem, several studies have been carried out including detecting phishing web pages using various features of web pages with various methods. Unfortunately, the use of several methods is not really effective because the design and evaluation are only too focused on the achievement of detection accuracy in research, but evaluation does not represent application in the real world. Whereas a security detection device should require effectiveness, good performance, and deployable. In this study the authors evaluated several methods and proposed rules-based applications that can detect phishing more efficiently.

The number of organisational cybersecurity threats continues to increase every year as technology advances. All too often, organisations assume that implementing systems security measures like firewalls and anti-virus software will eradicate cyber threats. However, even the most robust security systems are vulnerable to threats. As advanced as machine learning cybersecurity technology is becoming, it cannot be solely relied upon to solve cyber threats. There are other forces that contribute to these threats that are many-a-times out of an organisation's control i.e., human behaviour. This research article aims to create an understanding of the trends in key cybersecurity management issues that have developed in the past five years in relation to human behaviour and machine learning. The methodology adopted to guide the synthesis of this review was a systematic literature review. The guidelines for conducting the review are presented in the review approach. The key cybersecurity management issues highlighted by the research includes risky security behaviours demonstrated by employees, social engineering, the current limitations present in machine learning insider threat detection, machine learning enhanced cyber threats, and the underinvestment challenges faced in the cybersecurity domain.

Malware detection is an important area of cyber security. Computer systems rely on malware detection applications to prevent malware attacks from succeeding. Malware detection is not a straightforward task, as new variants of malware are generated at an increasing rate. Machine learning (ML) has been utilised to generate predictive classification models to identify new malware variants which conventional malware detection methods may not detect. Machine learning, has however, been found to be vulnerable to different types of adversarial attacks, in which an attacker is able to negatively affect the classification ability of the ML model. Several defensive measures to prevent adversarial poisoning attacks have been developed, but they often rely on the use of a trusted clean dataset to help identify and remove adversarial examples from the training dataset. The defence in this paper does not require a trusted clean dataset, but instead, identifies intentional false negatives (zero day malware classified as benign) at the testing stage by examining the activation weights of the ML model. The defence was able to identify 94.07% of the successful targeted poisoning attacks.

Recommender systems in software engineering provide developers with a wide range of valuable items to help them complete their tasks. Among others, API recommender systems have gained momentum in recent years as they became more successful at suggesting API calls or code snippets. While these systems have proven to be effective in terms of prediction accuracy, there has been less attention for what concerns such recommenders’ resilience against adversarial attempts. In fact, by crafting the recommenders’ learning material, e.g., data from large open-source software (OSS) repositories, hostile users may succeed in injecting malicious data, putting at risk the software clients adopting API recommender systems. In this paper, we present an empirical investigation of adversarial machine learning techniques and their possible influence on recommender systems. The evaluation performed on three state-of-the-art API recommender systems reveals a worrying outcome: all of them are not immune to malicious data. The obtained result triggers the need for effective countermeasures to protect recommender systems against hostile attacks disguised in training data.

A new method for the detection of ransomware in an infected host is described and evaluated. The method utilizes data streams from on-board sensors to fingerprint the initiation of a ransomware infection. These sensor streams, which are common in modern computing systems, are used as a side channel for understanding the state of the system. It is shown that ransomware detection can be achieved in a rapid manner and that the use of slight, yet distinguishable changes in the physical state of a system as derived from a machine learning predictive model is an effective technique. A feature vector, consisting of various sensor outputs, is coupled with a detection criteria to predict the binary state of ransomware present versus normal operation. An advantage of this approach is that previously unknown or zero-day version s of ransomware are vulnerable to this detection method since no apriori knowledge of the malware characteristics are required. Experiments are carried out with a variety of different system loads and with different encryption methods used during a ransomware attack. Two test systems were utilized with one having a relatively low amount of available sensor data and the other having a relatively high amount of available sensor data. The average time for attack detection in the "sensor-rich" system was 7.79 seconds with an average Matthews correlation coefficient of 0.8905 for binary system state predictions regardless of encryption method and system load. The model flagged all attacks tested.

In recent years web applications that are hacked every day estimated to be 30 000, and in most cases, web developers or website owners do not even have enough knowledge about what is happening on their sites. Web hackers can use many attacks to gain entry or compromise legitimate web applications, they can also deceive people by using phishing sites to collect their sensitive and private information. In response to this, the need is raised to take proper measures to understand the risks and be aware of the vulnerabilities that may affect the website and hence the normal business flow. In the scope of this study, mitigations against the most common web application attacks are set, and the web administrator is provided with ways to detect phishing links which is a social engineering attack, the study also demonstrates the generation of web application logs that simplifies the process of analyzing the actions of abnormal users to show when behavior is out of bounds, out of scope, or against the rules. The methods of mitigation are accomplished by secure coding techniques and the methods for phishing link detection are performed by various machine learning algorithms and deep learning techniques. The developed application has been tested and evaluated against various attack scenarios, the outcomes obtained from the test process showed that the website had successfully mitigated these dangerous web application attacks, and for the detection of phishing links part, a comparison is made between different algorithms to find the best one, and the outcome of the best model gave 98% accuracy.

CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) is commonly utilized as a technology for avoiding attacks to Web sites by bots. State-of-the-art CAPTCHAs vary in difficulty based on the client's behavior, allowing for efficient bot detection without sacrificing simplicity. In this research, we focus on detecting bots by supervised machine learning from access-log time series in the past. We have analysed access logs to several Web services which are using a commercial cloud-based CAPTCHA service, Capy Puzzle CAPTCHA. Experiments show that bot detection in attacks over a month can be performed with high accuracy by precursory analysis of the access log in only the first day as training data. In addition, we have manually analyzed the data that are found to be False Positive in the discrimination results, and it is found that the proposed model actually detects access by bots, which had been overlooked in the first-stage manual discrimination of flags in preparation of training data.

This study shows the effectiveness of anomaly-based IDS using long short-term memory(LSTM) based on the newly developed dataset called UNSW-NB15 while considering root mean square error and mean absolute error as evaluation metrics for accuracy. For each attack, 80% and 90% of samples were used as LSTM inputs and trained this model while increasing epoch values. Furthermore, this model has predicted attack points by applying test data and produced possible attack points for each attack at the 3rd time frame against the actual attack point. However, in the case of an Exploit attack, the consecutive overlapping attacks happen, there was ambiguity in the interpretation of the numerical values calculated by the LSTM. We presented a methodology for training data with binary values using LSTM and evaluation with RMSE metrics throughout this study.

In the era of Internet of Things (IoT), the connection links are established from devices easily, which is vulnerable to insecure attacks from intruders, hence intrusion detection system in IoT is the need of an hour. One of the important thing for any organization is securing the confidential information and data from outside attacks as well as unauthorized access. There are many attempts made by the researchers to develop the strong intrusion detection system having high accuracy. These systems suffer from many disadvantages like unacceptable accuracy rates including high False Positive Rate (FPR) and high False Negative Rate (FNR), more execution time and failure rate. More of these system models are developed by using traditional machine learning techniques, which have performance limitations in terms of accuracy and timeliness both. These limitations can be overcome by using the deep learning techniques. Deep learning techniques have the capability to generate highly accurate results and are fault tolerant. Here, the intrusion detection model for IoT is designed by using the Taylor-Spider Monkey optimization (Taylor-SMO) which will be developed to train the Deep belief neural network (DBN) towards achieving an accurate intrusion detection model. The deep learning accuracy gets increased with increasing number of training data samples and testing data samples. The optimization based algorithm for training DBN helps to reduce the FPR and FNR in intrusion detection. The system will be implemented by using the NSL KDD dataset. Also, this model will be trained by using the samples from this dataset, before which feature extraction will be applied and only relevant set of attributes will be selected for model development. This approach can lead to better and satisfactory results in intrusion detection.

How to develop AI systems that can explain how they made decisions is one of the important and hot topics today. Inspired by the dual-process theory in psychology, this paper proposes a human-in-the-loop approach to develop System-2 AI that makes an inference logically and outputs interpretable explanation. Our proposed method first asks crowd workers to raise understandable features of objects of multiple classes and collect training data from the Internet to generate classifiers for the features. Logical decision rules with the set of generated classifiers can explain why each object is of a particular class. In our preliminary experiment, we applied our method to an image classification of Asian national flags and examined the effectiveness and issues of our method. In our future studies, we plan to combine the System-2 AI with System-1 AI (e.g., neural networks) to efficiently output decisions.

Distant supervision is a widely used data labeling method for relation extraction. While aligning knowledge base with the corpus, distant supervision leads to a mass of wrong labels which are defined as noise. The pattern-based denoising model has achieved great progress in selecting trustable sentences (instances). However, the writing of relation-specific patterns heavily relies on expert’s knowledge and is a high labor intensity work. To solve these problems, we propose a noise reduction framework, NOIR, to iteratively select trustable sentences with a little help of a human. Under the guidance of experts, the iterative process can avoid semantic drift. Besides, NOIR can help experts discover relation-specific tokens that are hard to think of. Experimental results on three real-world datasets show the effectiveness of the proposed method compared with state-of-the-art methods.

In the era of machine learning, mobile users are able to submit their symptoms to doctors at any time, anywhere for personal diagnosis. It is prevalent to exploit edge computing for real-time diagnosis services in order to reduce transmission latency. Although data-driven machine learning is powerful, it inevitably compromises privacy by relying on vast amounts of medical data to build a diagnostic model. Therefore, it is necessary to protect data privacy without accessing local data. However, the blossom has also been accompanied by various problems, i.e., the limitation of training data, vulnerabilities, and privacy concern. As a solution to these above challenges, in this paper, we design a lightweight privacy-preserving medical diagnosis mechanism on edge. Our method redesigns the extreme gradient boosting (XGBoost) model based on the edge-cloud model, which adopts encrypted model parameters instead of local data to reduce amounts of ciphertext computation to plaintext computation, thus realizing lightweight privacy preservation on resource-limited edges. Additionally, the proposed scheme is able to provide a secure diagnosis on edge while maintaining privacy to ensure an accurate and timely diagnosis. The proposed system with secure computation could securely construct the XGBoost model with lightweight overhead, and efficiently provide a medical diagnosis without privacy leakage. Our security analysis and experimental evaluation indicate the security, effectiveness, and efficiency of the proposed system.

Federated learning is a distributed learning technique where machine learning models are trained on client devices in which the local training data resides. The training is coordinated via a central server which is, typically, controlled by the intended owner of the resulting model. By avoiding the need to transport the training data to the central server, federated learning improves privacy and efficiency. But it raises the risk of model theft by clients because the resulting model is available on every client device. Even if the application software used for local training may attempt to prevent direct access to the model, a malicious client may bypass any such restrictions by reverse engineering the application software. Watermarking is a well-known deterrence method against model theft by providing the means for model owners to demonstrate ownership of their models. Several recent deep neural network (DNN) watermarking techniques use backdooring: training the models with additional mislabeled data. Backdooring requires full access to the training data and control of the training process. This is feasible when a single party trains the model in a centralized manner, but not in a federated learning setting where the training process and training data are distributed among several client devices. In this paper, we present WAFFLE, the first approach to watermark DNN models trained using federated learning. It introduces a retraining step at the server after each aggregation of local models into the global model. We show that WAFFLE efficiently embeds a resilient watermark into models incurring only negligible degradation in test accuracy (-0.17%), and does not require access to training data. We also introduce a novel technique to generate the backdoor used as a watermark. It outperforms prior techniques, imposing no communication, and low computational (+3.2%) overhead$^\textrm1$$^\textrm1$\$The research report version of this paper is also available in https://arxiv.org/abs/2008.07298, and the code for reproducing our work can be found at https://github.com/ssg-research/WAFFLE.

With the advance in artificial intelligence and high-dimensional data analysis, federated learning (FL) has emerged to allow distributed data providers to collaboratively learn without direct access to local sensitive data. However, limiting access to individual provider’s data inevitably incurs security issues. For instance, backdoor attacks, one of the most popular data poisoning attacks in FL, severely threaten the integrity and utility of the FL system. In particular, backdoor attacks launched by multiple collusive attackers, i.e., distributed backdoor attacks, can achieve high attack success rates and are hard to detect. Existing defensive approaches, like model inspection or model sanitization, often require to access a portion of local training data, which renders them inapplicable to the FL scenarios. Recently, the norm clipping approach is developed to effectively defend against distributed backdoor attacks in FL, which does not rely on local training data. However, we discover that adversaries can still bypass this defense scheme through robust training due to its unchanged norm clipping threshold. In this paper, we propose a novel defense scheme to resist distributed backdoor attacks in FL. Particularly, we first identify that the main reason for the failure of the norm clipping scheme is its fixed threshold in the training process, which cannot capture the dynamic nature of benign local updates during the global model’s convergence. Motivated by it, we devise a novel defense mechanism to dynamically adjust the norm clipping threshold of local updates. Moreover, we provide the convergence analysis of our defense scheme. By evaluating it on four non-IID public datasets, we observe that our defense scheme effectively can resist distributed backdoor attacks and ensure the global model’s convergence. Noticeably, our scheme reduces the attack success rates by 84.23% on average compared with existing defense schemes.

Among many application domains of machine learning in real-world settings, cyber security can benefit from more automated techniques to combat sophisticated adversaries. Modern network intrusion detection systems leverage machine learning models on network logs to proactively detect cyber attacks. However, the risk of adversarial attacks against machine learning used in these cyber settings is not fully explored. In this paper, we investigate poisoning attacks at training time against machine learning models in constrained cyber environments such as network intrusion detection; we also explore mitigations of such attacks based on training data sanitization. We consider the setting of poisoning availability attacks, in which an attacker can insert a set of poisoned samples at training time with the goal of degrading the accuracy of the deployed model. We design a white-box, realizable poisoning attack that reduced the original model accuracy from 95% to less than 50 % by generating mislabeled samples in close vicinity of a selected subset of training points. We also propose a novel Nested Training method as a defense against these attacks. Our defense includes a diversified ensemble of classifiers, each trained on a different subset of the training set. We use the disagreement of the classifiers' predictions as a data sanitization method, and show that an ensemble of 10 SVM classifiers is resilient to a large fraction of poisoning samples, up to 30% of the training data.

Anecdotal evidence suggests that Web-search engines, together with the Knowledge Graphs and Bases, such as YAGO [46], DBPedia [13], Freebase [16], Google Knowledge Graph [52] provide rapid access to most structured information on the Web. However, taking a closer look reveals a so called "knowledge gap" [18] that is largely in the dark. For example, a person searching for a relevant job opening has to spend at least 3 hours per week for several months [2] just searching job postings on numerous online job-search engines and the employer websites. The reason why this seemingly simple task cannot be completed by typing in a few keyword queries into a search-engine and getting all relevant results in seconds instead of hours is because access to structured data on the Web is still rudimentary. While searching for a job we have many parameters in mind, not just the job title, but also, usually location, salary range, remote work option, given a recent shift to hybrid work places, and many others. Ideally, we would like to write a SQL-style query, selecting all job postings satisfying our requirements, but it is currently impossible, because job postings (and all other) Web tables are structured in many different ways and scattered all over the Web. There is neither a Web-scale generalizable algorithm nor a system to locate and normalize all relevant tables in a category of interest from millions of sources.Here we describe and evaluate on a corpus having hundreds of millions of Web tables [39], a new scalable iterative training data generation algorithm, producing high quality training data required to train Deep- and Machine-learning models, capable of generalizing to Web scale. The models, trained on such en-riched training data efficiently deal with Web scale heterogeneity compared to poor generalization performance of models, trained without enrichment [20], [25], [38]. Such models are instrumental in bridging the knowledge gap for structured data on the Web.

In this study, we aim to find a method to autotag sentences specific to a domain. Our training data comprises short conversational sentences extracted from chat conversations between company's customer representatives and web site visitors. We manually tagged approximately 14 thousand visitor inputs into ten basic categories, which will later be used in a transformer-based language model with attention mechanisms for the ultimate goal of developing a chatbot application that can produce meaningful dialogue.We considered three different stateof- the-art models and reported their auto-tagging capabilities. We achieved the best performance with the bidirectional encoder representation from transformers (BERT) model. Implementation of the models used in these experiments can be cloned from our GitHub repository and tested for similar auto-tagging problems without much effort.

Generalizing automated medical image segmentation methods to new image domains is inherently difficult. We have previously developed a number of automated segmentation methods that perform at the level of human readers on images acquired under similar conditions to the original training data. We are interested in exploring techniques that will improve model generalization to new imaging domains. In this study we explore a method to limit the inherent bias of these models to intensity and textural information. Using a dataset of 100 T2-weighted MR images with fat-saturation, and 100 T2-weighted MR images without fat-saturation, we explore the use of neural style transfer to induce shape preference and improve model performance on the task of segmenting the kidneys in patients affected by polycystic kidney disease. We find that using neural style transfer images improves the average dice value by 0.2. In addition, visualizing individual network kernel responses highlights a drastic difference in the optimized networks. Biasing models to invoke shape preference is a promising approach to create methods that are more closely aligned with human perception.

Despite recent advances in deep learning approaches, haze mitigation in large satellite images is still a challenging problem. Due to amorphous nature of haze, object detection or image segmentation approaches are not applicable. Also it is practically infeasible to obtain ground truths for training. Bounded memory capacity of GPUs is another constraint that limits the size of image to be processed. In this paper, we propose a style transfer based neural network approach to mitigate haze in a large overhead imagery. The network is trained without paired ground truths; further, perception loss is added to restore vivid colors, enhance contrast and minimize artifacts. The paper also illustrates our use of multiple GPUs in a collective way to produce a single coherent clear image where each GPU dehazes different portions of a large hazy image.

Cyber-attacks detection in modern power systems is undoubtedly indispensable to enhance their resilience and guarantee the continuous production of electricity. As the number of attacks is very small compared to normal events, and attacks are unpredictable, it is not obvious to build a model for attacks. Here, only anomaly-free measurements are utilized to build a reference model for intrusion detection. Specifically, this study presents an unsupervised intrusion detection approach using the k-nearest neighbor algorithm and exponential smoothing monitoring scheme for uncovering attacks in modern power systems. Essentially, the k-nearest neighbor algorithm is implemented to compute the deviation between actual measurements and the faultless (training) data. Then, the exponential smoothing method is used to set up a detection decision-based kNN metric for anomaly detection. The proposed procedure has been tested to detect cyber-attacks in a two-line three-bus power transmission system. The proposed approach has been shown good detection performance.

This paper defines a set difference metric for comparing machine learning (ML) datasets and proposes the difference between datasets be a function of combinatorial coverage. We illustrate its utility for evaluating and predicting performance of ML models. Identifying and measuring differences between datasets is of significant value for ML problems, where the accuracy of the model is heavily dependent on the degree to which training data are sufficiently representative of data encountered in application. The method is illustrated for transfer learning without retraining, the problem of predicting performance of a model trained on one dataset and applied to another.

Speaker verification has been widely and successfully adopted in many mission-critical areas for user identification. The training of speaker verification requires a large amount of data, therefore users usually need to adopt third-party data (e.g., data from the Internet or third-party data company). This raises the question of whether adopting untrusted third-party data can pose a security threat. In this paper, we demonstrate that it is possible to inject the hidden backdoor for infecting speaker verification models by poisoning the training data. Specifically, we design a clustering-based attack scheme where poisoned samples from different clusters will contain different triggers (i.e., pre-defined utterances), based on our understanding of verification tasks. The infected models behave normally on benign samples, while attacker-specified unenrolled triggers will successfully pass the verification even if the attacker has no information about the enrolled speaker. We also demonstrate that existing back-door attacks cannot be directly adopted in attacking speaker verification. Our approach not only provides a new perspective for designing novel attacks, but also serves as a strong baseline for improving the robustness of verification methods. The code for reproducing main results is available at https://github.com/zhaitongqing233/Backdoor-attack-against-speaker-verification.