| Title | Anomaly Detection in Unstructured Logs Using Attention-based Bi-LSTM Network |
| Publication Type | Conference Paper |
| Year of Publication | 2021 |
| Authors | Yu, Dongqing, Hou, Xiaowei, Li, Ce, Lv, Qiujian, Wang, Yan, Li, Ning |
| Conference Name | 2021 7th IEEE International Conference on Network Intelligence and Digital Content (IC-NIDC) |
| Keywords | anomaly detection, Conferences, Deep Learning, feature extraction, ICS Anomaly Detection, Linux, Log sequence, Natural languages, Production, pubcrawl, resilience, Resiliency, Runtime, Scalability, Semantics, Unstructured log |
| Abstract | System logs record valuable information about the runtime status of IT systems. Therefore, system logs are a naturally excellent source of information for anomaly detection. Most of the existing studies on log-based anomaly detection construct a detection model to identify anomalous logs. Generally, the model treats historical logs as natural language sequences and learns the normal patterns from normal log sequences, and detects deviations from normal patterns as anomalies. However, the majority of existing methods focus on sequential and quantitative information and ignore semantic information hidden in log sequence so that they are inefficient in anomaly detection. In this paper, we propose a novel framework for automatically detecting log anomalies by utilizing an attention-based Bi-LSTM model. To demonstrate the effectiveness of our proposed model, we evaluate the performance on a public production log dataset. Extensive experimental results show that the proposed approach outperforms all comparison methods for anomaly detection. |
| DOI | 10.1109/IC-NIDC54101.2021.9660476 |
| Citation Key | yu_anomaly_2021 |
Anomaly Detection in Unstructured Logs Using Attention-based Bi-LSTM Network