Biblio

Automated tests play an important role in software evolution because they can rapidly detect faults introduced during changes. In practice, code-coverage metrics are often used as criteria to evaluate the effectiveness of test suites with focus on regression faults. However, code coverage only expresses which portion of a system has been executed by tests, but not how effective the tests actually are in detecting regression faults. Our goal was to evaluate the validity of code coverage as a measure for test effectiveness. To do so, we conducted an empirical study in which we applied an extreme mutation testing approach to analyze the tests of open-source projects written in Java. We assessed the ratio of pseudo-tested methods (those tested in a way such that faults would not be detected) to all covered methods and judged their impact on the software project. The results show that the ratio of pseudo-tested methods is acceptable for unit tests but not for system tests (that execute large portions of the whole system). Therefore, we conclude that the coverage metric is only a valid effectiveness indicator for unit tests.

Agile methodologies are becoming widespread in modern software development. However, due to a lack of safety assurance activities, agile methods are criticized for being inadequate for the development of safe software. Safety analysis and safety verification are complementary methods for safety assurance. Yet, both usually rely on traditional, waterfall-like processes. Therefore, it is strongly needed to integrate an appropriate safety analysis approach into agile software development processes driving architecture design and verify the safe design at the code level. This paper presents a novel agile process model "S-Scrum" based on the existing development process "Safe Scrum" and extended by a safety analysis method and a safety verification approach based on STPA (System-Theoretic Process Analysis). The proposed agile development process S-Scrum can be separated into three parts: (1) performing safety-guided design by STPA inside each sprint. (2) Verifying safety requirements at the code level by using model checking. (3) Replacing traditional RAMS (Reliability, Availability, Maintainability, Safety) validation on the final product by STPA safety analysis. We adopt other aspects from the original Safe Scrum. Finally, the feasibility of S-Scrum is illustrated with the example of an airbag system.

There are many techniques to improve software quality. One is using automatic static analysis tools. We have observed, however, that despite the low-cost help they offer, these tools are underused and often discourage beginners. There is evidence that personality traits influence the perceived usability of a software. Thus, to support beginners better, we need to understand how the workflow of people with different prevalent personality traits using these tools varies. For this purpose, we observed users' solution strategies and correlated them with their prevalent personality traits in an exploratory study with student participants within a controlled experiment. We gathered data by screen capturing and chat protocols as well as a Big Five personality traits test. We found strong correlations between particular personality traits and different strategies of removing the findings of static code analysis as well as between personality and tool utilization. Based on that, we offer take-away improvement suggestions. Our results imply that developers should be aware of these solution strategies and use this information to build tools that are more appealing to people with different prevalent personality traits.