Biblio

Found 792 results

Filters: First Letter Of Last Name is O  [Clear All Filters]
2018-05-17
2015-07-01
2023-06-29
Campbell, Donal, Rafferty, Ciara, Khalid, Ayesha, O'Neill, Maire.  2022.  Acceleration of Post Quantum Digital Signature Scheme CRYSTALS-Dilithium on Reconfigurable Hardware. 2022 32nd International Conference on Field-Programmable Logic and Applications (FPL). :462–463.
This research investigates efficient architectures for the implementation of the CRYSTALS-Dilithium post-quantum digital signature scheme on reconfigurable hardware, in terms of speed, memory usage, power consumption and resource utilisation. Post quantum digital signature schemes involve a significant computational effort, making efficient hardware accelerators an important contributor to future adoption of schemes. This is work in progress, comprising the establishment of a comprehensive test environment for operational profiling, and the investigation of the use of novel architectures to achieve optimal performance.
ISSN: 1946-1488
2023-05-12
Ornik, Melkior, Bouvier, Jean-Baptiste.  2022.  Assured System-Level Resilience for Guaranteed Disaster Response. 2022 IEEE International Smart Cities Conference (ISC2). :1–4.
Resilience of urban infrastructure to sudden, system-wide, potentially catastrophic events is a critical need across domains. The growing connectivity of infrastructure, including its cyber-physical components which can be controlled in real time, offers an attractive path towards rapid adaptation to adverse events and adjustment of system objectives. However, existing work in the field often offers disjoint approaches that respond to particular scenarios. On the other hand, abstract work on control of complex systems focuses on attempting to adapt to the changes in the system dynamics or environment, but without understanding that the system may simply not be able to perform its original task after an adverse event. To address this challenge, this programmatic paper proposes a vision for a new paradigm of infrastructure resilience. Such a framework treats infrastructure across domains through a unified theory of controlled dynamical systems, but remains cognizant of the lack of knowledge about the system following a widespread adverse event and aims to identify the system's fundamental limits. As a result, it will enable the infrastructure operator to assess and assure system performance following an adverse event, even if the exact nature of the event is not yet known. Building off ongoing work on assured resilience of control systems, in this paper we identify promising early results, challenges that motivate the development of resilience theory for infrastructure system, and possible paths forward for the proposed effort.
ISSN: 2687-8860
2023-07-20
Khokhlov, Igor, Okutan, Ahmet, Bryla, Ryan, Simmons, Steven, Mirakhorli, Mehdi.  2022.  Automated Extraction of Software Names from Vulnerability Reports using LSTM and Expert System. 2022 IEEE 29th Annual Software Technology Conference (STC). :125—134.
Software vulnerabilities are closely monitored by the security community to timely address the security and privacy issues in software systems. Before a vulnerability is published by vulnerability management systems, it needs to be characterized to highlight its unique attributes, including affected software products and versions, to help security professionals prioritize their patches. Associating product names and versions with disclosed vulnerabilities may require a labor-intensive process that may delay their publication and fix, and thereby give attackers more time to exploit them. This work proposes a machine learning method to extract software product names and versions from unstructured CVE descriptions automatically. It uses Word2Vec and Char2Vec models to create context-aware features from CVE descriptions and uses these features to train a Named Entity Recognition (NER) model using bidirectional Long short-term memory (LSTM) networks. Based on the attributes of the product names and versions in previously published CVE descriptions, we created a set of Expert System (ES) rules to refine the predictions of the NER model and improve the performance of the developed method. Experiment results on real-life CVE examples indicate that using the trained NER model and the set of ES rules, software names and versions in unstructured CVE descriptions could be identified with F-Measure values above 0.95.
2023-02-03
Ouamour, S., Sayoud, H..  2022.  Computational Identification of Author Style on Electronic Libraries - Case of Lexical Features. 2022 5th International Symposium on Informatics and its Applications (ISIA). :1–4.
In the present work, we intend to present a thorough study developed on a digital library, called HAT corpus, for a purpose of authorship attribution. Thus, a dataset of 300 documents that are written by 100 different authors, was extracted from the web digital library and processed for a task of author style analysis. All the documents are related to the travel topic and written in Arabic. Basically, three important rules in stylometry should be respected: the minimum document size, the same topic for all documents and the same genre too. In this work, we made a particular effort to respect those conditions seriously during the corpus preparation. That is, three lexical features: Fixed-length words, Rare words and Suffixes are used and evaluated by using a centroid based Manhattan distance. The used identification approach shows interesting results with an accuracy of about 0.94.
2023-06-09
Wang, Shuangbao Paul, Arafin, Md Tanvir, Osuagwu, Onyema, Wandji, Ketchiozo.  2022.  Cyber Threat Analysis and Trustworthy Artificial Intelligence. 2022 6th International Conference on Cryptography, Security and Privacy (CSP). :86—90.
Cyber threats can cause severe damage to computing infrastructure and systems as well as data breaches that make sensitive data vulnerable to attackers and adversaries. It is therefore imperative to discover those threats and stop them before bad actors penetrating into the information systems.Threats hunting algorithms based on machine learning have shown great advantage over classical methods. Reinforcement learning models are getting more accurate for identifying not only signature-based but also behavior-based threats. Quantum mechanics brings a new dimension in improving classification speed with exponential advantage. The accuracy of the AI/ML algorithms could be affected by many factors, from algorithm, data, to prejudicial, or even intentional. As a result, AI/ML applications need to be non-biased and trustworthy.In this research, we developed a machine learning-based cyber threat detection and assessment tool. It uses two-stage (both unsupervised and supervised learning) analyzing method on 822,226 log data recorded from a web server on AWS cloud. The results show the algorithm has the ability to identify the threats with high confidence.
2023-06-22
Ho, Samson, Reddy, Achyut, Venkatesan, Sridhar, Izmailov, Rauf, Chadha, Ritu, Oprea, Alina.  2022.  Data Sanitization Approach to Mitigate Clean-Label Attacks Against Malware Detection Systems. MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM). :993–998.
Machine learning (ML) models are increasingly being used in the development of Malware Detection Systems. Existing research in this area primarily focuses on developing new architectures and feature representation techniques to improve the accuracy of the model. However, recent studies have shown that existing state-of-the art techniques are vulnerable to adversarial machine learning (AML) attacks. Among those, data poisoning attacks have been identified as a top concern for ML practitioners. A recent study on clean-label poisoning attacks in which an adversary intentionally crafts training samples in order for the model to learn a backdoor watermark was shown to degrade the performance of state-of-the-art classifiers. Defenses against such poisoning attacks have been largely under-explored. We investigate a recently proposed clean-label poisoning attack and leverage an ensemble-based Nested Training technique to remove most of the poisoned samples from a poisoned training dataset. Our technique leverages the relatively large sensitivity of poisoned samples to feature noise that disproportionately affects the accuracy of a backdoored model. In particular, we show that for two state-of-the art architectures trained on the EMBER dataset affected by the clean-label attack, the Nested Training approach improves the accuracy of backdoor malware samples from 3.42% to 93.2%. We also show that samples produced by the clean-label attack often successfully evade malware classification even when the classifier is not poisoned during training. However, even in such scenarios, our Nested Training technique can mitigate the effect of such clean-label-based evasion attacks by recovering the model's accuracy of malware detection from 3.57% to 93.2%.
ISSN: 2155-7586
2023-06-30
Mimoto, Tomoaki, Hashimoto, Masayuki, Yokoyama, Hiroyuki, Nakamura, Toru, Isohara, Takamasa, Kojima, Ryosuke, Hasegawa, Aki, Okuno, Yasushi.  2022.  Differential Privacy under Incalculable Sensitivity. 2022 6th International Conference on Cryptography, Security and Privacy (CSP). :27–31.
Differential privacy mechanisms have been proposed to guarantee the privacy of individuals in various types of statistical information. When constructing a probabilistic mechanism to satisfy differential privacy, it is necessary to consider the impact of an arbitrary record on its statistics, i.e., sensitivity, but there are situations where sensitivity is difficult to derive. In this paper, we first summarize the situations in which it is difficult to derive sensitivity in general, and then propose a definition equivalent to the conventional definition of differential privacy to deal with them. This definition considers neighboring datasets as in the conventional definition. Therefore, known differential privacy mechanisms can be applied. Next, as an example of the difficulty in deriving sensitivity, we focus on the t-test, a basic tool in statistical analysis, and show that a concrete differential privacy mechanism can be constructed in practice. Our proposed definition can be treated in the same way as the conventional differential privacy definition, and can be applied to cases where it is difficult to derive sensitivity.
2023-02-03
Oldal, Laura Gulyás, Kertész, Gábor.  2022.  Evaluation of Deep Learning-based Authorship Attribution Methods on Hungarian Texts. 2022 IEEE 10th Jubilee International Conference on Computational Cybernetics and Cyber-Medical Systems (ICCC). :000161–000166.
The range of text analysis methods in the field of natural language processing (NLP) has become more and more extensive thanks to the increasing computational resources of the 21st century. As a result, many deep learning-based solutions have been proposed for the purpose of authorship attribution, as they offer more flexibility and automated feature extraction compared to traditional statistical methods. A number of solutions have appeared for the attribution of English texts, however, the number of methods designed for Hungarian language is extremely small. Hungarian is a morphologically rich language, sentence formation is flexible and the alphabet is different from other languages. Furthermore, a language specific POS tagger, pretrained word embeddings, dependency parser, etc. are required. As a result, methods designed for other languages cannot be directly applied on Hungarian texts. In this paper, we review deep learning-based authorship attribution methods for English texts and offer techniques for the adaptation of these solutions to Hungarian language. As a part of the paper, we collected a new dataset consisting of Hungarian literary works of 15 authors. In addition, we extensively evaluate the implemented methods on the new dataset.
2023-07-18
El Makkaoui, Khalid, Lamriji, Youssef, Ouahbi, Ibrahim, Nabil, Omayma, Bouzahra, Anas, Beni-Hssane, Abderrahim.  2022.  Fast Modular Exponentiation Methods for Public-Key Cryptography. 2022 5th International Conference on Advanced Communication Technologies and Networking (CommNet). :1—6.
Modular exponentiation (ME) is a complex operation for several public-key cryptosystems (PKCs). Moreover, ME is expensive for resource-constrained devices in terms of computation time and energy consumption, especially when the exponent is large. ME is defined as the task of raising an integer x to power k and reducing the result modulo some integer n. Several methods to calculate ME have been proposed. In this paper, we present the efficient ME methods. We then implement the methods using different security levels of RSA keys on a Raspberry Pi. Finally, we give the fast ME method.
2023-07-14
Reis, Lúcio H. A., de Oliveira, Marcela T., Olabarriaga, Sílvia D..  2022.  Fine-grained Encryption for Secure Research Data Sharing. 2022 IEEE 35th International Symposium on Computer-Based Medical Systems (CBMS). :465–470.
Research data sharing requires provision of adequate security. The requirements for data privacy are extremely demanding for medical data that is reused for research purposes. To address these requirements, the research institutions must implement adequate security measurements, and this demands large effort and costs to do it properly. The usage of adequate access controls and data encryption are key approaches to effectively protect research data confidentiality; however, the management of the encryption keys is challenging. There are novel mechanisms that can be explored for managing access to the encryption keys and encrypted files. These mechanisms guarantee that data are accessed by authorised users and that auditing is possible. In this paper we explore these mechanisms to implement a secure research medical data sharing system. In the proposed system, the research data are stored on a secure cloud system. The data are partitioned into subsets, each one encrypted with a unique key. After the authorisation process, researchers are given rights to use one or more of the keys and to selectively access and decrypt parts of the dataset. Our proposed solution offers automated fine-grain access control to research data, saving time and work usually made manually. Moreover, it maximises and fortifies users' trust in data sharing through secure clouds solutions. We present an initial evaluation and conclude with a discussion about the limitations, open research questions and future work around this challenging topic.
ISSN: 2372-9198
2023-03-03
Mishra, Ruby, Okade, Manish, Mahapatra, Kamalakanta.  2022.  FPGA based High Throughput Substitution Box Architectures for Lightweight Block Ciphers. 2022 IEEE International Conference on Public Key Infrastructure and its Applications (PKIA). :1–7.
This paper explores high throughput architectures for the substitution modules, which are an integral component of encryption algorithms. The security algorithms chosen belong to the category of lightweight crypto-primitives suitable for pervasive computing. The focus of this work is on the implementation of encryption algorithms on hardware platforms to improve speed and facilitate optimization in the area and power consumption of the design. In this work, the architecture for the encryption algorithms' substitution box (S-box) is modified using switching circuits (i.e., MUX-based) along with a logic generator and included in the overall cipher design. The modified architectures exhibit high throughput and consume less energy in comparison to the state-of-the-art designs. The percentage increase in throughput or maximum frequency differs according to the chosen algorithms discussed elaborately in this paper. The evaluation of various metrics specific to the design are executed at RFID-specific frequency so that they can be deployed in an IoT environment. The designs are mainly simulated and compared on Nexys4 DDR FPGA platform, along with a few other FPGAs, to meet similar design and implementation environments for a fair comparison. The application of the proposed S-box modification is explored for the healthcare scenario with promising results.
2023-08-17
Otta, Soumya Prakash, Panda, Subhrakanta, Hota, Chittaranjan.  2022.  Identity Management with Blockchain : Indian Migrant Workers Prospective. 2022 IEEE Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI). :1—6.
The agricultural sector and other Micro, Small, and Medium Enterprises in India operate with more than 90% migrant workers searching for better employment opportunities far away from their native places. However, inherent challenges are far more for the migrant workers, most prominently their Identity. To the best of our knowledge, available literature lacks a comprehensive study on identity management components for user privacy and data protection mechanisms in identity management architecture. Self-Sovereign Identity is regarded as a new evolution in digital identity management systems. Blockchain technology and distributed ledgers bring us closer to realizing an ideal Self-Sovereign Identity system. This paper proposes a novel solution to address identity issues being faced by migrant workers. It also gives a holistic, coherent, and mutually beneficial Identity Management Solution for the migrant workforce in the Indian perspective towards e-Governance and Digital India.
2023-01-20
Núñez, Ivonne, Cano, Elia, Rovetto, Carlos, Ojo-Gonzalez, Karina, Smolarz, Andrzej, Saldana-Barrios, Juan Jose.  2022.  Key technologies applied to the optimization of smart grid systems based on the Internet of Things: A Review. 2022 V Congreso Internacional en Inteligencia Ambiental, Ingeniería de Software y Salud Electrónica y Móvil (AmITIC). :1—8.
This article describes an analysis of the key technologies currently applied to improve the quality, efficiency, safety and sustainability of Smart Grid systems and identifies the tools to optimize them and possible gaps in this area, considering the different energy sources, distributed generation, microgrids and energy consumption and production capacity. The research was conducted with a qualitative methodological approach, where the literature review was carried out with studies published from 2019 to 2022, in five (5) databases following the selection of studies recommended by the PRISMA guide. Of the five hundred and four (504) publications identified, ten (10) studies provided insight into the technological trends that are impacting this scenario, namely: Internet of Things, Big Data, Edge Computing, Artificial Intelligence and Blockchain. It is concluded that to obtain the best performance within Smart Grids, it is necessary to have the maximum synergy between these technologies, since this union will enable the application of advanced smart digital technology solutions to energy generation and distribution operations, thus allowing to conquer a new level of optimization.
2023-02-17
Yang, Jingcong, Xia, Qi, Gao, Jianbin, Obiri, Isaac Amankona, Sun, Yushan, Yang, Wenwu.  2022.  A Lightweight Scalable Blockchain Architecture for IoT Devices. 2022 IEEE 5th International Conference on Electronics Technology (ICET). :1014–1018.
With the development of Internet of Things (IoT) technology, the transaction behavior of IoT devices has gradually increased, which also brings the problem of transaction data security and transaction processing efficiency. As one of the research hotspots in the field of data security, blockchain technology has been widely applied in the maintenance of transaction records and the construction of financial payment systems. However, the proportion of microtransactions in the Internet of Things poses challenges to the coupling of blockchain and IoT devices. This paper proposes a three-party scalable architecture based on “IoT device-edge server-blockchain”. In view of the characteristics of micropayment, the verification mechanism of the execution results of the off-chain transaction is designed, and the bridge node is designed in the off-chain architecture, which ensures the finality of the blockchain to the transaction. According to system evaluation, this scalable architecture improves the processing efficiency of micropayments on blockchain, while ensuring its decentration equal to that of blockchain. Compared with other blockchain-based IoT device payment schemes, our architecture is more excellent in activity.
ISSN: 2768-6515
2023-02-03
Suzumura, Toyotaro, Sugiki, Akiyoshi, Takizawa, Hiroyuki, Imakura, Akira, Nakamura, Hiroshi, Taura, Kenjiro, Kudoh, Tomohiro, Hanawa, Toshihiro, Sekiya, Yuji, Kobayashi, Hiroki et al..  2022.  mdx: A Cloud Platform for Supporting Data Science and Cross-Disciplinary Research Collaborations. 2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech). :1–7.
The growing amount of data and advances in data science have created a need for a new kind of cloud platform that provides users with flexibility, strong security, and the ability to couple with supercomputers and edge devices through high-performance networks. We have built such a nation-wide cloud platform, called "mdx" to meet this need. The mdx platform's virtualization service, jointly operated by 9 national universities and 2 national research institutes in Japan, launched in 2021, and more features are in development. Currently mdx is used by researchers in a wide variety of domains, including materials informatics, geo-spatial information science, life science, astronomical science, economics, social science, and computer science. This paper provides an overview of the mdx platform, details the motivation for its development, reports its current status, and outlines its future plans.
2023-05-12
Matsubayashi, Masaru, Koyama, Takuma, Tanaka, Masashi, Okano, Yasushi, Miyajima, Asami.  2022.  Message Source Identification in Controller Area Network by Utilizing Diagnostic Communications and an Intrusion Detection System. 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall). :1–6.
International regulations specified in WP.29 and international standards specified in ISO/SAE 21434 require security operations such as cyberattack detection and incident responses to protect vehicles from cyberattacks. To meet these requirements, many vehicle manufacturers are planning to install Intrusion Detection Systems (IDSs) in the Controller Area Network (CAN), which is a primary component of in-vehicle networks, in the coming years. Besides, many vehicle manufacturers and information security companies are developing technologies to identify attack paths related to IDS alerts to respond to cyberattacks appropriately and quickly. To develop the IDSs and the technologies to identify attack paths, it is essential to grasp normal communications performed on in-vehicle networks. Thus, our study aims to develop a technology that can easily grasp normal communications performed on in-vehicle networks. In this paper, we propose the first message source identification method that easily identifies CAN-IDs used by each Electronic Control Unit (ECU) connected to the CAN for message transmissions. We realize the proposed method by utilizing diagnostic communications and an IDS installed in the CAN (CAN-IDS). We evaluate the proposed method using an ECU installed in an actual vehicle and four kinds of simulated CAN-IDSs based on typical existing intrusion detection methods for the CAN. The evaluation results show that the proposed method can identify the CAN-ID used by the ECU for CAN message transmissions if a suitable simulated CAN-IDS for the proposed method is connected to the vehicle.
ISSN: 2577-2465
2023-06-23
Özdel, Süleyman, Damla Ateş, Pelin, Ateş, Çağatay, Koca, Mutlu, Anarım, Emin.  2022.  Network Anomaly Detection with Payload-based Analysis. 2022 30th Signal Processing and Communications Applications Conference (SIU). :1–4.
Network attacks become more complicated with the improvement of technology. Traditional statistical methods may be insufficient in detecting constantly evolving network attack. For this reason, the usage of payload-based deep packet inspection methods is very significant in detecting attack flows before they damage the system. In the proposed method, features are extracted from the byte distributions in the payload and these features are provided to characterize the flows more deeply by using N-Gram analysis methods. The proposed procedure has been tested on IDS 2012 and 2017 datasets, which are widely used in the literature.
ISSN: 2165-0608
2023-03-17
Alam, Md Shah, Hossain, Sarkar Marshia, Oluoch, Jared, Kim, Junghwan.  2022.  A Novel Secure Physical Layer Key Generation Method in Connected and Autonomous Vehicles (CAVs). 2022 IEEE Conference on Communications and Network Security (CNS). :1–6.
A novel secure physical layer key generation method for Connected and Autonomous Vehicles (CAVs) against an attacker is proposed under fading and Additive White Gaussian Noise (AWGN). In the proposed method, a random sequence key is added to the demodulated sequence to generate a unique pre-shared key (PSK) to enhance security. Extensive computer simulation results proved that an attacker cannot extract the same legitimate PSK generated by the received vehicle even if identical fading and AWGN parameters are used both for the legitimate vehicle and attacker.
2023-02-17
Mohammadi, Ali Akbar, Hussain, Rasheed, Oracevic, Alma, Kazmi, Syed Muhammad Ahsan Raza, Hussain, Fatima, Aloqaily, Moayad, Son, Junggab.  2022.  A Novel TCP/IP Header Hijacking Attack on SDN. IEEE INFOCOM 2022 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). :1–2.
Middlebox is primarily used in Software-Defined Network (SDN) to enhance operational performance, policy compliance, and security operations. Therefore, security of the middlebox itself is essential because incorrect use of the middlebox can cause severe cybersecurity problems for SDN. Existing attacks against middleboxes in SDN (for instance, middleboxbypass attack) use methods such as cloned tags from the previous packets to justify that the middlebox has processed the injected packet. Flowcloak as the latest solution to defeat such an attack creates a defence using a tag by computing the hash of certain parts of the packet header. However, the security mechanisms proposed to mitigate these attacks are compromise-able since all parts of the packet header can be imitated, leaving the middleboxes insecure. To demonstrate our claim, we introduce a novel attack against SDN middleboxes by hijacking TCP/IP headers. The attack uses crafted TCP/IP headers to receive the tags and signatures and successfully bypasses the middleboxes.
Tilloo, Pallavi, Parron, Jesse, Obidat, Omar, Zhu, Michelle, Wang, Weitian.  2022.  A POMDP-based Robot-Human Trust Model for Human-Robot Collaboration. 2022 12th International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). :1009–1014.
Trust is a cognitive ability that can be dependent on behavioral consistency. In this paper, a partially observable Markov Decision Process (POMDP)-based computational robot-human trust model is proposed for hand-over tasks in human-robot collaborative contexts. The robot's trust in its human partner is evaluated based on the human behavior estimates and object detection during the hand-over task. The human-robot hand-over process is parameterized as a partially observable Markov Decision Process. The proposed approach is verified in real-world human-robot collaborative tasks. Results show that our approach can be successfully applied to human-robot hand-over tasks to achieve high efficiency, reduce redundant robot movements, and realize predictability and mutual understanding of the task.
ISSN: 2642-6633
2023-09-18
Oshio, Kei, Takada, Satoshi, Han, Chansu, Tanaka, Akira, Takeuchi, Jun'ichi.  2022.  Poster: Flexible Function Estimation of IoT Malware Using Graph Embedding Technique. 2022 IEEE Symposium on Computers and Communications (ISCC). :1—3.
Most IoT malware is variants generated by editing and reusing parts of the functions based on publicly available source codes. In our previous study, we proposed a method to estimate the functions of a specimen using the Function Call Sequence Graph (FCSG), which is a directed graph of execution sequence of function calls. In the FCSG-based method, the subgraph corresponding to a malware functionality is manually created and called a signature-FSCG. The specimens with the signature-FSCG are expected to have the corresponding functionality. However, this method cannot detect the specimens with a slightly different subgraph from the signature-FSCG. This paper found that these specimens were supposed to have the same functionality for a signature-FSCG. These specimens need more flexible signature matching, and we propose a graph embedding technique to realize it.
2023-05-12
Borg, Markus, Bengtsson, Johan, Österling, Harald, Hagelborn, Alexander, Gagner, Isabella, Tomaszewski, Piotr.  2022.  Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice. 2022 IEEE/ACM 1st International Conference on AI Engineering – Software Engineering for AI (CAIN). :22–32.
Due to the migration megatrend, efficient and effective second-language acquisition is vital. One proposed solution involves AI-enabled conversational agents for person-centered interactive language practice. We present results from ongoing action research targeting quality assurance of proprietary generative dialog models trained for virtual job interviews. The action team elicited a set of 38 requirements for which we designed corresponding automated test cases for 15 of particular interest to the evolving solution. Our results show that six of the test case designs can detect meaningful differences between candidate models. While quality assurance of natural language processing applications is complex, we provide initial steps toward an automated framework for machine learning model selection in the context of an evolving conversational agent. Future work will focus on model selection in an MLOps setting.