Systematization of Knowledge from Intrusion Detection Models - January 2016

Public Audience
Purpose: To highlight project progress. Information is generally at a higher level which is accessible to the interested public. All information contained in the report (regions 1-3) is a Government Deliverable/CDRL.

PI(s):  Huaiyu Dai, Andy Meneely
Researchers:

Xiaofan He, Yufan Huang, Richeng Jin, Nuthan Munaiah, Kevin Campusano Gonzalez

HARD PROBLEM(S) ADDRESSED

• Security Metrics and Models - The project aims to establish common criteria for evaluating and systematizing knowledge contributed by research on intrusion detection models.
• Resilient Architectures - Robust intrusion detection models serve to make large systems more resilient to attack.
• Scalability and Composability - Intrusion detection models deal with large data sets every day, so scale is always a significant concern.
• Humans - A key aspect of intrusion detection is interpreting the output and acting upon it, which inherently involves humans. Furthermore, intrusion detection models are ultimately simulations of human behavior.

PUBLICATIONS
Report papers written as a results of this research. If accepted by or submitted to a journal, which journal. If presented at a conference, which conference.

• Xiaofan He, Huaiyu Dai, Peng Ning, Rudra Dutta.  2015.  Dynamic IDS Configuration in the Presence of Intruder Type Uncertainty. IEEE Global Conference on Communications (GLOBECOM).

ACCOMPLISHMENT HIGHLIGHTS

• We have collected new data on 8,654 intrusion detection studies and have found that the community lacks consistent use of validation metrics. For example, out of those data, we estimate that only 35% of those studies report false positives (or Type II) errors. What is more rare is when researchers report both accuracy metrics and performance metrics such as speed and memory usage. We also saw similar problems with benchmarks - most data are custom benchmarks, which undermines replication. Our factorial study explores eight different categories of metrics and approximately 40 performance metrics that IDS studies use (or fail to use).

• We continued our study on IDS collaboration. The problem is being formulated as a two-layer game: the first layer game models the interplay between each IDS and its corresponding attackers, while the second-layer game models the collaboration among the IDSs. The algorithm development is ongoing, and it is expected that the sharing of resources among IDSs can increase the detection rates and reduce the false alarms.