Biblio

With the rapid development of the Internet, network traffic is becoming more complex and diverse. At the same time, malicious traffic is growing. This seriously threatens the security of networks and information. However, the current DPI (Deep Packet Inspect) engine based on x86 architecture is slow in monitoring speed, which cannot meet the needs. Generally, two factors affect the detection rate: CPU and memory; The efficiency of data packet acquisition, and multi regular expression matching. Under these circumstances, this paper presents an efficient implementation of the DPI engine based on a generic x86 platform. DPDK is used as the platform of network data packets acquisition and processing. Using the multi-queue of the NIC (network interface controller) and the customized symmetric RSS key, the network traffic is divided and reorganized in the form of conversation. The core of traffic identification is hyperscan, which uses a flow pattern to match the packets load of a single conversation efficiently. It greatly reduces memory requirements. The method makes full use of the system resources and takes into account the advantages of high efficiency of hardware implementation. And it has a remarkable improvement in the efficiency of recognition.

Based on Storm, a distributed, reliable, fault-tolerant real-time data stream processing system, we propose a recognition system of web intrusion detection. The system is based on machine learning, feature selection algorithm by TF-IDF(Term Frequency–Inverse Document Frequency) and the optimised cosine similarity algorithm, at low false positive rate and a higher detection rate of attacks and malicious behavior in real-time to protect the security of user data. From comparative analysis of experiments we find that the system for intrusion recognition rate and false positive rate has improved to some extent, it can be better to complete the intrusion detection work.