Visible to the public Development of Methodology Guidelines for Security ResearchConflict Detection Enabled

Submitted by nzahan on Tue, 03/16/2021 - 2:04pm

PI(s), Co-PI(s), Researchers:

PI: Jeffrey Carver

HARD PROBLEM(S) ADDRESSED
This refers to Hard Problems, released November 2012.

PUBLICATIONS
Still in progress, none submitted/published yet

KEY HIGHLIGHTS
Each effort should submit one or two specific highlights. Each item should include a paragraph or two along with a citation if available. Write as if for the general reader of IEEE S&P.
The purpose of the highlights is to give our immediate sponsors a body of evidence that the funding they are providing (in the framework of the SoS lablet model) is delivering results that "more than justify" the investment they are making.

We completed and submitted a manuscript to the IEEE Security and Privacy magazine. This manuscript presents examples from published cyber security literature to illustrate good examples of publishing cyber security research in a scientifically rigorous and valid manner. The goal of this work is to help further key attributes of scientific progress, including replication, meta-analysis, and theory building. This manuscript describes 12 examples from published literature to illustrate good practices in reporting: research questions/objectives, methodologies, analysis techniques, and research conclusions.

Another focus of this quarter was analysis of qualitative data gathered from interviews with cyber security experts. The interviews discussed the unique requirements of reporting cyber security research and the kinds of information required to report cyber security research in a scientifically rigorous and valid manner. We have completed the coding of the qualitative interview data. We have also begun analysis of this data by forming an initial set of guidelines for publishing scientifically rigorous and valid cyber security research. We submitted and had accepted a poster for HotSoS'21, which presents the initial findings from the analysis and a first version of publication guidelines. This poster will allow us to interact with the community to gather feedback on the first version of these publication guidelines.

We have also been working with the Program Chairs of the previous edition of HotSoS to obtain consent from program committee members to access their anonymized reviews of HotSoS paper. The goal of this work is to analyze the types of comments provided by reviewers to gain further insight into the types of information important for science of security. We are still in the process of obtaining consent and anticipate beginning the analysis in the next quarter.


COMMUNITY ENGAGEMENTS

  • We have begun engaging with members of prior HoTSoS organizing and program committees to obtain consent for our comment review process study.
  • The Good Examples article is based on past SoS work and showcases papers listed as "Notable SoS Papers" (https://cps-vo.org/node/61173).

EDUCATIONAL ADVANCES:

None

Groups: