| Title | Hybrid Deep Neural Networks to Infer State Models of Black-Box Systems |
| Publication Type | Conference Paper |
| Year of Publication | 2020 |
| Authors | Mashhadi, M. J., Hemmati, H. |
| Conference Name | 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE) |
| Date Published | sep |
| Keywords | 888 random recent system-level test cases, aerospace computing, anomaly detection, automated software engineering tasks, black box encryption, Black-box Model Inference, black-box systems, black-box techniques, black-box third-party libraries, C code, Classification algorithms, composability, convolutional layers, convolutional neural network, Deep Learning, detection techniques, entire source code, existing dynamic model inference techniques, hybrid deep neural network, infer state models, inference mechanisms, inferred states, inferring behavior model, Instruments, learning (artificial intelligence), Metrics, neural nets, pattern classification, program comprehension, program testing, pubcrawl, recurrent layers, recurrent neural network, Resiliency, run-time traces, running software system, software engineering, Specification Mining, state change points, state classification algorithm, Task Analysis, Testing, Time measurement, time series, Time series analysis, traditional time series change, UAV auto-pilot solution, white-box |
| Abstract | Inferring behavior model of a running software system is quite useful for several automated software engineering tasks, such as program comprehension, anomaly detection, and testing. Most existing dynamic model inference techniques are white-box, i.e., they require source code to be instrumented to get run-time traces. However, in many systems, instrumenting the entire source code is not possible (e.g., when using black-box third-party libraries) or might be very costly. Unfortunately, most black-box techniques that detect states over time are either univariate, or make assumptions on the data distribution, or have limited power for learning over a long period of past behavior. To overcome the above issues, in this paper, we propose a hybrid deep neural network that accepts as input a set of time series, one per input/output signal of the system, and applies a set of convolutional and recurrent layers to learn the non-linear correlations between signals and the patterns, over time. We have applied our approach on a real UAV auto-pilot solution from our industry partner with half a million lines of C code. We ran 888 random recent system-level test cases and inferred states, over time. Our comparison with several traditional time series change point detection techniques showed that our approach improves their performance by up to 102%, in terms of finding state change points, measured by F1 score. We also showed that our state classification algorithm provides on average 90.45% F1 score, which improves traditional classification algorithms by up to 17%. |
| Citation Key | mashhadi_hybrid_2020 |
Hybrid Deep Neural Networks to Infer State Models of Black-Box Systems