Improving Vulnerability Detection Measurement: [Test Suites and Software Security Assurance]
| Title | Improving Vulnerability Detection Measurement: [Test Suites and Software Security Assurance] |
| Publication Type | Conference Paper |
| Year of Publication | 2016 |
| Authors | Hoole, Alexander M., Traore, Issa, Delaitre, Aurelien, de Oliveira, Charles |
| Conference Name | Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering |
| Publisher | ACM |
| Conference Location | New York, NY, USA |
| ISBN Number | 978-1-4503-3691-8 |
| Keywords | Collaboration, dynamic analysis, governance, Government, Human Behavior, Metrics, policy, policy-based governance, pubcrawl, Resiliency, security metrics, Security weakness, security weaknesses, static analysis, test suites, Vulnerability, weakness |
| Abstract | The Software Assurance Metrics and Tool Evaluation (SAMATE) project at the National Institute of Standards and Technology (NIST) has created the Software Assurance Reference Dataset (SARD) to provide researchers and software security assurance tool developers with a set of known security flaws. As part of an empirical evaluation of a runtime monitoring framework, two test suites were executed and monitored, revealing deficiencies which led to a collaboration with the NIST SAMATE team to provide replacements. Test Suites 45 and 46 are analyzed, discussed, and updated to improve accuracy, consistency, preciseness, and automation. Empirical results show metrics such as recall, precision, and F-Measure are all impacted by invalid base assumptions regarding the test suites. |
| URL | http://doi.acm.org/10.1145/2915970.2915994 |
| DOI | 10.1145/2915970.2915994 |
| Citation Key | hoole_improving_2016 |