Security Profile of Fedora

The process of software development and evolution has proven difficult to improve. For example, well documented security issues such as SQL injection (SQLi), after more than a decade, still top most vulnerability lists. Quantitative security process and quality metrics are often subdued due to lack of time and resources. Security problems are hard to quantify and even harder to predict or relate to any process improvement activity. The goal of this thesis is to assess usefulness of "classical" software reliability engineering (SRE) models in the context of open source software security, the conditions under which they may be useful, and the information that they can provide with respect to the security quality of a software product. We start with security problem reports for open source Fedora series of software releases.We illustrate how one can learn from normal operational profile about the non-operational processes related to security problems. One aspect is classification of security problems based on the human traits that contribute to the injection of problems into code, whether due to poor practices or limited knowledge (epistemic errors), or due to random accidental events (aleatoric errors). Knowing the distribution aids in development of an attack profile. In the case of Fedora, the distribution of security problems found post-release was consistent across four different releases of the software. The security problem discovery rate appears to be roughly constant but much lower than the initial non-security problem discovery rate. Previous work has shown that non-operational testing can help accelerate and focus the problem discovery rate and that it can be successfully modeled.We find that some classical reliability models can be used with success to estimate the residual number of security problems, and through that provide a measure of the security characteristics of the software. We propose an agile software testing process that combines operational and non-operational (or attack related) testing with the intent of finding more security problems faster.