An Adaptive Real-Time Architecture for Zero-Day Threat Detection

Attackers create new threats and constantly change their behavior to mislead security systems. In this paper, we propose an adaptive threat detection architecture that trains its detection models in real time. The major contributions of the proposed architecture are: i) gather data about zero-day attacks and attacker behavior using honeypots in the network; ii) process data in real time and achieve high processing throughput through detection schemes implemented with stream processing technology; iii) use of two real datasets to evaluate our detection schemes, the first from a major network operator in Brazil and the other created in our lab; iv) design and development of adaptive detection schemes including both online trained supervised classification schemes that update their parameters in real time and learn zero-day threats from the honeypots, and online trained unsupervised anomaly detection schemes that model legitimate user behavior and adapt to changes. The performance evaluation results show that proposed architecture maintains an excellent trade-off between threat detection and false positive rates and achieves high classification accuracy of more than 90%, even with legitimate behavior changes and zero-day threats.