A Big Data Architecture for Large Scale Security Monitoring
| Title | A Big Data Architecture for Large Scale Security Monitoring |
| Publication Type | Conference Paper |
| Year of Publication | 2014 |
| Authors | Marchal, S., Xiuyan Jiang, State, R., Engel, T. |
| Conference Name | Big Data (BigData Congress), 2014 IEEE International Congress on |
| Date Published | June |
| Keywords | Big Data, Big Data architecture, computer network security, Correlation, data correlation schemes, data exploitation, data mining, digital forensics, Distributed databases, distributed system, DNS data, forensic analysis, Hadoop, honeypot data, HTTP traffic, IP networks, large scale security monitoring, local enterprise networks, Monitoring, NetFlow records, network intrusion detection, network intrusion prevention, network traffic, scalable distributed data management, scalable distributed data storage, security, Spark, storage management, telecommunication traffic, transport protocols |
| Abstract | Network traffic is a rich source of information for security monitoring. However the increasing volume of data to treat raises issues, rendering holistic analysis of network traffic difficult. In this paper we propose a solution to cope with the tremendous amount of data to analyse for security monitoring perspectives. We introduce an architecture dedicated to security monitoring of local enterprise networks. The application domain of such a system is mainly network intrusion detection and prevention, but can be used as well for forensic analysis. This architecture integrates two systems, one dedicated to scalable distributed data storage and management and the other dedicated to data exploitation. DNS data, NetFlow records, HTTP traffic and honeypot data are mined and correlated in a distributed system that leverages state of the art big data solution. Data correlation schemes are proposed and their performance are evaluated against several well-known big data framework including Hadoop and Spark. |
| URL | https://ieeexplore.ieee.org/document/6906761/ |
| DOI | 10.1109/BigData.Congress.2014.18 |
| Citation Key | 6906761 |
- IP networks
- transport protocols
- telecommunication traffic
- storage management
- Spark
- security
- scalable distributed data storage
- scalable distributed data management
- network traffic
- network intrusion prevention
- network intrusion detection
- NetFlow records
- Monitoring
- local enterprise networks
- large scale security monitoring
- Big Data
- HTTP traffic
- honeypot data
- Hadoop
- forensic analysis
- DNS data
- distributed system
- Distributed databases
- Digital Forensics
- Data mining
- data exploitation
- data correlation schemes
- Correlation
- computer network security
- Big Data architecture