Biblio

OS kernel is the core part of the operating system, and it plays an important role for OS resource management. A popular way to compromise OS kernel is through a kernel rootkit (i.e., malicious kernel module). Once a rootkit is loaded into the kernel space, it can carry out arbitrary malicious operations with high privilege. To defeat kernel rootkits, many approaches have been proposed in the past few years. However, existing methods suffer from some limitations: 1) most methods focus on user-mode rootkit detection; 2) some methods are limited to detect obfuscated kernel modules; and 3) some methods introduce significant performance overhead. To address these problems, we propose VKRD, a kernel rootkit detection system based on the hardware assisted virtualization technology. Compared with previous methods, VKRD can provide a transparent and an efficient execution environment for the target kernel module to reveal its run-time behavior. To select the important run-time features for training our detection models, we utilize the TF-IDF method. By combining the hardware assisted virtualization and machine learning techniques, our kernel rootkit detection solution could be potentially applied in the cloud environment. The experiments show that our system can detect windows kernel rootkits with high accuracy and moderate performance cost.

The return-oriented programming(ROP) attack has been a common access to exploit software vulnerabilities in the modern operating system(OS). An attacker can execute arbitrary code with the aid of ROP despite security mechanisms are involved in OS. In order to mitigate ROP attack, defense mechanisms are also drawn researchers' attention. Besides, research on the benign use of ROP become a hot spot in recent years, since ROP has a perfect resistance to static analysis, which can be adapted to hide some important code. The results in benign use also benefit from a low overhead on program size. The paper discusses the concepts of ROP attack as well as extended ROP attack in recent years. Corresponding defense mechanisms based on randomization, frequency, and control flow integrity are analyzed as well, besides, we also analyzed limitations in this defense mechanisms. Later, we discussed the benign use of ROP in steganography, code integrity verification, and software watermarking, which showed the significant promotion by adopting ROP. At the end of this paper, we looked into the development of ROP attack, the future of possible mitigation strategies and the potential for benign use.

Security is an important requirement of every reactive system of the smart gird. The devices connected to the smart system in smart grid are exhaustively used to provide digital information to outside world. The security of such a system is an essential requirement. The most important component of such smart systems is Operating System (OS). This paper mainly focuses on the security of OS by incorporating Access Control Mechanism (ACM) which will improve the efficiency of the smart system. The formal methods use applied mathematics for modelling and analysing of smart systems. In the proposed work Formal Security Analysis (FSA) is used with model checking and hence it helped to prove the security of smart systems. When an Operating System (OS) takes into consideration, it never comes to a halt state. In the proposed work a Transition System (TS) is designed and the desired rules of security are provided by using Linear Temporal Logics (LTL). Unlike other propositional and predicate logic, LTL can model reactive systems with a prediction for the future state of the systems. In the proposed work, Simple Promela Interpreter (SPIN) is used as a model checker that takes LTL and TS of the system as input. Hence it is possible to derive the Büchi automaton from LTL logics and that provides traces of both successful and erroneous computations. Comparison of Büchi automaton with the transition behaviour of the OS will provide the details of security violation in the system. Validation of automaton operations on infinite computational sequences verify that whether systems are provably secure or not. Hence the proposed formal security analysis will provably ensures the security of smart systems in the area of smart grid applications.

We present AVAMAT: AntiVirus and Malware Analysis Tool - a tool for analysing the malware detection capabilities of AntiVirus (AV) products running on different operating system (OS) platforms. Even though similar tools are available, such as VirusTotal and MetaDefender, they have several limitations, which motivated the creation of our own tool. With AVAMAT we are able to analyse not only whether an AV detects a malware, but also at what stage of inspection does it detect it and on what OS. AVAMAT enables experimental campaigns to answer various research questions, ranging from the detection capabilities of AVs on OSs, to optimal ways in which AVs could be combined to improve malware detection capabilities.