| Title | Combinatorial Testing Metrics for Machine Learning |
| Publication Type | Conference Paper |
| Year of Publication | 2021 |
| Authors | Lanus, Erin, Freeman, Laura J., Richard Kuhn, D., Kacker, Raghu N. |
| Conference Name | 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) |
| Keywords | combinatorial testing, Conferences, Data models, machine learning, Measurement, Measurement and Metrics Testing, Metrics, operating envelopes, Predictive models, pubcrawl, test set selection, Training data, transfer learning |
| Abstract | This paper defines a set difference metric for comparing machine learning (ML) datasets and proposes the difference between datasets be a function of combinatorial coverage. We illustrate its utility for evaluating and predicting performance of ML models. Identifying and measuring differences between datasets is of significant value for ML problems, where the accuracy of the model is heavily dependent on the degree to which training data are sufficiently representative of data encountered in application. The method is illustrated for transfer learning without retraining, the problem of predicting performance of a model trained on one dataset and applied to another. |
| DOI | 10.1109/ICSTW52544.2021.00025 |
| Citation Key | lanus_combinatorial_2021 |
Combinatorial Testing Metrics for Machine Learning