NCSU SoS Lablet Research Methods, Community Development and Support - July 2016

Public Audience
Purpose: To highlight project progress. Information is generally at a higher level which is accessible to the interested public. All information contained in the report (regions 1-3) is a Government Deliverable/CDRL.

PI(s):  Jeff Carver, Ehab Al-Shaer, Lindsey McGowen, Laurie Williams, Jon Stallings, Rahul Pandita
Researchers: Morgan Burcham (UA), Mahran Al-Zyoud (UA)

PROBLEM ADDRESSED

• Community Development - The goal is to build an extended and vibrant interdisciplinary community of science of security researchers, research methodologists, and practitioners (Carver, Williams).
• Community Resources - To create and maintain a repository of defensible scientific methods for security research (Carver, Williams).
• Oversight for the Application of Defensible Scientific Research Methodologies - To encourage the application of scientifically defensible research through various methods of consultation and feedback (Carver).
• Usable Data Sharing - To enable open, efficient, and secure sharing of data and experimental results for experimentation among SoS researchers (Al-Shaer).

PUBLICATIONS
Report papers written as a result of this research. If accepted by or submitted to a journal, which journal. If presented at a conference, which conference.

ACCOMPLISHMENT HIGHLIGHTS

• The analysis of the ACM CCS 2015 proceedings in the context of Science of Security, which we submitted to ACM CCS 2016, found that there was low evidence of study replication and the use of Case Study was more frequent than the use of Experiments as an evaluation approach.  We also found that papers frequently omitted details about the study procedures including Research Objectives, rationale for selection of cases, and threats to validity.
• Hosted the Summer 2016 SoSL NCSU Summer Workshop