Visible to the public A framework for the automation of testing computer vision systems

TitleA framework for the automation of testing computer vision systems
Publication TypeConference Paper
Year of Publication2021
AuthorsWotawa, Franz, Klampfl, Lorenz, Jahaj, Ledio
Conference Name2021 IEEE/ACM International Conference on Automation of Software Test (AST)
KeywordsAutomation, compositionality, expandability, image recognition, Measurement, pubcrawl, quality assurance, Resiliency, Software, software reliability, surveillance, test case generation, testing image classifiers, testing vision software
AbstractVision systems, i.e., systems that enable the detection and tracking of objects in images, have gained substantial importance over the past decades. They are used in quality assurance applications, e.g., for finding surface defects in products during manufacturing, surveillance, but also automated driving, requiring reliable behavior. Interestingly, there is only little work on quality assurance and especially testing of vision systems in general. In this paper, we contribute to the area of testing vision software, and present a framework for the automated generation of tests for systems based on vision and image recognition with the focus on easy usage, uniform usability and expandability. The framework makes use of existing libraries for modifying the original images and to obtain similarities between the original and modified images. We show how such a framework can be used for testing a particular industrial application on identifying defects on riblet surfaces and present preliminary results from the image classification domain.
DOI10.1109/AST52587.2021.00023
Citation Keywotawa_framework_2021