Biblio

Even though build automation tools help to reduce errors and rapid releases of software changes, use of build automation tools is not widespread amongst software practitioners. Software practitioners perceive build automation tools as complex, which can hinder the adoption of these tools. How well founded such perception is, can be determined by systematic exploration of adoption factors that influence usage of build automation tools. The goal of this paper is to aid software practitioners in increasing their usage of build automation tools by identifying the adoption factors that influence usage of these tools. We conducted a survey to empirically identify the adoption factors that influence usage of build automation tools. We obtained survey responses from 268 software professionals who work at NestedApps, Red Hat, as well as contribute to open source software. We observe that adoption factors related to complexity do not have the strongest influence on usage of build automation tools. Instead, we observe compatibility-related adoption factors, such as adjustment with existing tools, and adjustment with practitioner's existing workflow, to have influence on usage of build automation tools with greater importance. Findings from our paper suggest that usage of build automation tools might increase if: build automation tools fit well with practitioners' existing workflow and tool usage; and usage of build automation tools are made more visible among practitioners' peers.

Android applications pose security and privacy risks for end-users. These risks are often quantified by performing dynamic analysis and permission analysis of the Android applications after release. Prediction of security and privacy risks associated with Android applications at early stages of application development, e.g. when the developer (s) are writing the code of the application, might help Android application developers in releasing applications to end-users that have less security and privacy risk. The goal of this paper is to aid Android application developers in assessing the security and privacy risk associated with Android applications by using static code metrics as predictors. In our paper, we consider security and privacy risk of Android application as how susceptible the application is to leaking private information of end-users and to releasing vulnerabilities. We investigate how effectively static code metrics that are extracted from the source code of Android applications, can be used to predict security and privacy risk of Android applications. We collected 21 static code metrics of 1,407 Android applications, and use the collected static code metrics to predict security and privacy risk of the applications. As the oracle of security and privacy risk, we used Androrisk, a tool that quantifies the amount of security and privacy risk of an Android application using analysis of Android permissions and dynamic analysis. To accomplish our goal, we used statistical learners such as, radial-based support vector machine (r-SVM). For r-SVM, we observe a precision of 0.83. Findings from our paper suggest that with proper selection of static code metrics, r-SVM can be used effectively to predict security and privacy risk of Android applications.

Scientific advancement is fueled by solid fundamental research, followed by replication, meta-analysis, and theory building. To support such advancement, researchers and government agencies have been working towards a "science of security". As in other sciences, security science requires high-quality fundamental research addressing important problems and reporting approaches that capture the information necessary for replication, meta-analysis, and theory building. The goal of this paper is to aid security researchers in establishing a baseline of the state of scientific reporting in security through an analysis of indicators of scientific research as reported in top security conferences, specifically the 2015 ACM CCS and 2016 IEEE S&P proceedings. To conduct this analysis, we employed a series of rubrics to analyze the completeness of information reported in papers relative to the type of evaluation used (e.g. empirical study, proof, discussion). Our findings indicated some important information is often missing from papers, including explicit documentation of research objectives and the threats to validity. Our findings show a relatively small number of replications reported in the literature. We hope that this initial analysis will serve as a baseline against which we can measure the advancement of the science of security.

Automated cyber attacks tend to be schedule and resource limited. The primary progress metric is often “coverage” of pre-determined “known” vulnerabilities that may not have been patched, along with possible zero-day exploits (if such exist). We present and discuss a hypergeometric process model that describes such attack patterns. We used web request signatures from the logs of a production web server to assess the applicability of the model.

We have been studying extension of the classical Software Reliability Engineering (SRE) methodology into the security space. We combine “classical” reliability modeling, when applied to reported vulnerabilities found under “normal” operational profile conditions, with safety oriented fault management processes. We illustrate with open source Fedora software.

Our initial results appear to indicate that generation of a repeatable automated test-strategy that would explicitly cover the “top 25” security problems may help considerably – eliminating perhaps as much as 50% of the field observable problems. However, genuine aleatoric and more process oriented incomplete analysis and design flaws remain. While we have made some progress in identifying focus areas, a number of questions remain, and we continue working on them.

Can software reliability models be used to assess software security? One of the issues is that security problems are relatively rare under “normal” operational profiles, while “classical” reliability models may not be suitable for use in attack conditions. We investigated a range of Fedora open source software security problems to see if some of the basic assumptions behind software reliability growth models hold for discovery of security problems in non-attack situations. We find that in some cases, under “normal” operational use, security problem detection process may be described as a Poisson process. In those cases, we can use appropriate classical software reliability growth models to assess “security reliability” of that software in non-attack situations.We analyzed security problem discovery rate for RedHat Fedora. We find that security problems are relatively rare, their rate of discovery appears to be relatively constant under “normal” (non-attack) conditions. Discovery process often appears to satisfy Poisson assumption opening doors to use of classical reliability models. We illustrated using Yamada S-shaped model fit to v15 that in some cases such models may be effective in predicting the number of remaining security problems, and thus may offer a way of assessing security “quality” of the software product (although not necessarily its behavior under an attack).

In our previous work we showed that for Fedora, under normal operational conditions, security problem discovery appears to be a random process. While in the case of Fedora, and a number of other open source products, classical reliability models can be adapted to estimate the number of residual security problems under “normal” operational usage (not attacks), the predictive ability of the model is lower for security faults due to the rarity of security events and because there appears to be no real security reliability growth. The ratio of security to non-security faults is an indicator that the process needs improving, but it also may be leveraged to assess vulnerability profile of a release and possibly guide testing of its next version. We manually analyzed randomly sampled problems for four different versions of Fedora and classified them into security vulnerability categories. We also analyzed the distribution of these problems over the software’s lifespan and we found that they exhibit a symmetry which can perhaps be used in estimating the number of residual security problems in the software. Based on our work, we believe that an approach to vulnerability elimination based on a combination of “classical” and some non-operational “bounded” high-assurance testing along the lines discussed in may yield good vulnerability elimination results, as well as a way of estimating vulnerability level of a release. Classical SRE methods, metrics and models can be used to track both non-security and security problem detection under normal operational profile. We can then model the reliability growth, if any, and estimate the number of residual faults by estimating the lower and upper bounds on the total number of faults of a certain type. In production, there may be a simpler alternative. Just count the vulnerabilities and project over the next period assuming constant vulnerability discovery rate. In testing phase, to accelerate the process, one might leverage collected vulnerability information to generate non-operational test-cases aimed at vulnerability categories. The observed distributions of security problems reported under normal “operational” usage appear to support the above approach – i.e., what is learned say in the first x weeks can them be leveraged in selecting test cases in the next stage. Similarly, what is learned about a product Y weeks after its release may be very indicative of its vulnerability profile for the rest of its life given the assumption of constant vulnerability discovery rate.

Software patterns are created with the goal of capturing expert
knowledge so it can be efficiently and effectively shared with the
software development community. However, patterns in practice
may or may not achieve these goals. Empirical studies of the use
of software patterns can help in providing deeper insight into
whether these goals have been met. The objective of this paper is
to aid researchers in designing empirical studies of software
patterns by summarizing the study designs of software patterns
available in the literature. The important components of these
study designs include the evaluation criteria and how the patterns
are presented to study participants. We select and analyze 19
distinct empirical studies and identify 17 independent variables in
three different categories (participants demographics; pattern
presentation; problem presentation). We also extract 10 evaluation
criteria with 23 associated observable measures. Additionally, by
synthesizing the reported observations, we identify challenges
faced during study execution. Provision of multiple domainspecific
examples of pattern application and tool support to assist
in pattern selection are helpful for the study participants in
understanding and completing the study task. Capturing data
regarding the cognitive processes of participants can provide
insights into the findings of the study.

Research shows that commonly accepted security requirements are not generally applied in practice. Instead of relying on requirements checklists, security experts rely on their expertise and background knowledge to identify security vulnerabilities. To understand the gap between available checklists and practice, we conducted a series of interviews to encode the decision-making process of security experts and novices during security requirements analysis. Participants were asked to analyze two types of artifacts: source code, and network diagrams for vulnerabilities and to apply a requirements checklist to mitigate some of those vulnerabilities. We framed our study using Situation Awareness-a cognitive theory from psychology-to elicit responses that we later analyzed using coding theory and grounded analysis. We report our preliminary results of analyzing two interviews that reveal possible decision-making patterns that could characterize how analysts perceive, comprehend and project future threats which leads them to decide upon requirements and their specifications, in addition, to how experts use assumptions to overcome ambiguity in specifications. Our goal is to build a model that researchers can use to evaluate their security requirements methods against how experts transition through different situation awareness levels in their decision-making process.