Biblio

Due to the migration megatrend, efficient and effective second-language acquisition is vital. One proposed solution involves AI-enabled conversational agents for person-centered interactive language practice. We present results from ongoing action research targeting quality assurance of proprietary generative dialog models trained for virtual job interviews. The action team elicited a set of 38 requirements for which we designed corresponding automated test cases for 15 of particular interest to the evolving solution. Our results show that six of the test case designs can detect meaningful differences between candidate models. While quality assurance of natural language processing applications is complex, we provide initial steps toward an automated framework for machine learning model selection in the context of an evolving conversational agent. Future work will focus on model selection in an MLOps setting.

Software quality assurance (SQA) is a means and practice of monitoring the software engineering processes and methods used in a project to ensure proper quality of the software. It encompasses the entire software development life-cycle, including requirements engineering, software design, coding, source code reviews, software configuration management, testing , release management, software deployment and software integration. It is organized into goals, commitments, abilities, activities, measurements, verification and validation. In this talk, we will mainly focus on the testing activity part of the software development life-cycle. Its main objective is checking that software is satisfying a set of quality properties that are identified by the "ISO/IEC 25010:2011 System and Software Quality Model" standard [1] .

Synthetic static code analysis test suites are important to test the basic functionality of tools. We present a framework that uses different source code patterns to generate Cross Site Scripting and SQL injection test cases. A decision tree is used to determine if the test cases are vulnerable. The test cases are split into two test suites. The first test suite contains 258,432 test cases that have influence on the decision trees. The second test suite contains 20 vulnerable test cases with different data flow patterns. The test cases are scanned with two commercial static code analysis tools to show that they can be used to benchmark and identify problems of static code analysis tools. Expert interviews confirm that the decision tree is a solid way to determine the vulnerable test cases and that the test suites are relevant.

Source code security audit is an effective technique to deal with security vulnerabilities and software bugs. As one kind of white-box testing approaches, it can effectively help developers eliminate defects in the code. However, it suffers from performance issues. In this paper, we propose an incremental checking mechanism which enables fast source code security audits. And we conduct comprehensive experiments to verify the effectiveness of our approach.

Privacy protection is a vital part of information security. However, the excessive collections and uses of personal information have intensified in the area of mobile apps (applications). To comprehend the current situation of APP personal information security problem of APP, this paper uses a combined approach of static analysis technology, dynamic analysis technology, and manual review to detect and analyze the installed file of mobile apps. 40 mobile apps are detected as experimental samples. The results demonstrate that this combined approach can effectively detect various issues of personal information security problem in mobile apps. Statistics analysis of the experimental results demonstrate that mobile apps have outstanding problems in some aspects of personal information security such as privacy policy, permission application, information collection, data storage, etc.

The semaphore mechanism is an important part of the embedded operating system. Therefore, it is very necessary to ensure its safety. Traditional software testing methods are difficult to ensure 100% coverage of the program. Therefore, it is necessary to adopt a formal verfication method which proves the correctness of the program theoretically. This paper proposes a proof framework based on the theorem proof tool Coq: modeling the semaphore mechanism, extracting important properties from the requirement documents, and finally verifying that the semaphore mechanism can meet these properties, which means the correctness of the semaphore mechanism is proved and also illustrates the feasibility of the verification framework proposed in this paper, which lays a foundation for the verification of other modules of operating systems.

Mutation-based taint inference (MTI) is a novel technique for taint analysis. Compared with traditional techniques that track propagations of taint tags, MTI infers a variable is tainted if its values change due to input mutations, which is lightweight and conceptually sound. However, there are 3 challenges to its efficiency and scalability: (1) it cannot efficiently record variable values to monitor their changes; (2) it consumes a large amount of memory monitoring variable values, especially on complex programs; and (3) its excessive memory overhead leads to a low hit ratio of CPU cache, which slows down the speed of taint inference. This paper presents an efficient and scalable solution named HashMTI. We first explain the above challenges based on 4 observations. Motivated by these challenges, we propose a hash record scheme to efficiently monitor changes in variable values and significantly reduce the memory overhead. The scheme is based on our specially selected and optimized hash functions that possess 3 crucial properties. Moreover, we propose the DoubleMutation strategy, which applies additional mutations to mitigate the limitation of the hash record and detect more taint information. We implemented a prototype of HashMTI and evaluated it on 18 real-world programs and 4 LAVA-M programs. Compared with the baseline OrigMTI, HashMTI significantly reduces the overhead while having similar accuracy. It achieves a speedup of 2.5X to 23.5X and consumes little memory which is on average 70.4 times less than that of OrigMTI.

Internet of Things (IoT) apps provide great convenience but exposes us to new safety threats. Unlike traditional software systems, threats may emerge from the joint behavior of multiple apps. While prior studies use handcrafted safety and security policies to detect these threats, these policies may not anticipate all usages of the devices and apps in a smart home, causing false alarms. In this study, we propose to use the technique of mining sandboxes for securing an IoT environment. After a set of behaviors are analyzed from a bundle of apps and devices, a sandbox is deployed, which enforces that previously unseen behaviors are disallowed. Hence, the execution of malicious behavior, introduced from software updates or obscured through methods to hinder program analysis, is blocked.While sandbox mining techniques have been proposed for Android apps, we show and discuss why they are insufficient for detecting malicious behavior in a more complex IoT system. We prototype IoTBox to address these limitations. IoTBox explores behavior through a formal model of a smart home. In our empirical evaluation to detect malicious code changes, we find that IoTBox achieves substantially higher precision and recall compared to existing techniques for mining sandboxes.

Development of large-scale and complex software systems requires multiple teams, including software development teams, domain experts, user representatives, and other project stakeholders, to work collaboratively to achieve software development goals. These teams rely on the use of agreed software development processes, knowledge management tools, and communication channels collaboratively in the software development project. Software testing is an important and complicated process due to reasons such as difficulties in achieving testing goals with the given time constraint, absence of efficient data sharing policies, vague testing acceptance criteria at various levels of testing, and lack of trusted coordination among the teams involved in software testing. The efficiency of the software testing relies on efficient, reliable, and trusted information sharing among these teams. Existing approaches to software testing for collaborative software development use centralized or decentralize tools for software testing, knowledge management, and communication channels. Existing approaches have the limitations of centralized authority, a single point of failure/compromise, lack of automatic requirement compliance checking and transparency in information sharing, and lack of unified data sharing policy, and reliable knowledge management repositories for sharing and storing past software testing artifacts and data. In this paper, a software testing approach for collaborative software development using private blockchain is presented, and the desirable properties of private blockchain, such as distributed data management, tamper-resistance, auditability and automatic requirement compliance checking, are incorporated to greatly improve the quality of software testing for collaborative software development.

Modern mobile operating systems store a lot of excessive information that can be used against its owner or organization, like a call history or various system logs. This article describes a universal way of preventing any mobile operating system or application from saving its data in device's internal storage without reducing their functionality. The goal of this work is creation of a software that solves the described problem and works on the bootloading stage. A general algorithm of the designed software, along with its main solutions and requirements, is presented in this paper. Hardware requirement, software testing results and general applications of this software are also listed in this paper.

The "aging" phenomenon occurs after the long-term running of software, with the fault rate rising and running efficiency dropping. As there is no corresponding testing type for this phenomenon among conventional software tests, "software runtime accumulative testing" is proposed. Through analyzing several examples of software aging causing serious accidents, software is placed in the system environment required for running and the occurrence mechanism of software aging is analyzed. In addition, corresponding testing contents and recommended testing methods are designed with regard to all factors causing software aging, and the testing process and key points of testing requirement analysis for carrying out runtime accumulative testing are summarized, thereby providing a method and guidance for carrying out "software runtime accumulative testing" in software engineering.

Testing which is an indispensable part of software engineering is itself an art and science which emerged as a discipline over a period. On testing, if defects are found, testers diminish the risk by providing the awareness of defects and solutions to deal with them before release. If testing does not find any defects, testing assure that under certain conditions the system functions correctly. To guarantee that enough testing has been done, major risk areas need to be tested. We have to identify the risks, analyse and control them. We need to categorize the risk items to decide the extent of testing to be covered. Also, Implementation of structured metrics is lagging in software testing. Efficient metrics are necessary to evaluate, manage the testing process and make testing a part of engineering discipline. This paper proposes the usage of risk based testing using FMEA technique and provides an ideal set of metrics which provides a way to ensure effective testing process.

Currently, the complexity of software quality and testing is increasing exponentially with a huge number of challenges knocking doors, especially when testing a mission-critical application in banking and other critical domains, or the new technology trends with decentralized and nonintegrated testing tools. From practical experience, software testing has become costly and more effort-intensive with unlimited scope. This thesis promotes the Scalable Quality and Testing Lab (SQTL), it's a centralized quality and testing platform, which integrates a powerful manual, automation and business intelligence tools. SQTL helps quality engineers (QE) effectively organize, manage and control all testing activities in one centralized lab, starting from creating test cases, then executing different testing types such as web, security and others. And finally, ending with analyzing and displaying all testing activities result in an interactive dashboard, which allows QE to forecast new bugs especially those related to security. The centralized SQTL is to empower QE during the testing cycle, help them to achieve a greater level of software quality in minimum time, effort and cost, and decrease defect density metric.

Artificial intelligence (AI) software has high algorithm complexity, and the scale and dimension of the input and output parameters are high, and the test oracle isn't explicit. These features make a lot of difficulties for the design of test cases. This paper proposes an AI software testing method based on scene deductive approach. It models the input, output parameters and the environment, uses the random algorithm to generate the inputs of the test cases, then use the algorithm of deductive approach to make the software testing automatically, and use the test assertions to verify the results of the test. After description of the theory, this paper uses intelligent tracking car as an example to illustrate the application of this method and the problems needing attention. In the end, the paper describes the shortcoming of this method and the future research directions.

In recent years, data published and shared with third parties to develop artificial intelligence (AI) tools and services has significantly increased. When there are regulatory or internal requirements regarding privacy of data, anonymization techniques are used to maintain privacy by transforming the data. The side-effect is that the anonymization may lead to useless data to train and test the AI because it is highly dependent on the quality of the data. To overcome this problem, we propose a test-driven anonymization approach for artificial intelligence tools. The approach tests different anonymization efforts to achieve a trade-off in terms of privacy (non-functional quality) and functional suitability of the artificial intelligence technique (functional quality). The approach has been validated by means of two real-life datasets in the domains of healthcare and health insurance. Each of these datasets is anonymized with several privacy protections and then used to train classification AIs. The results show how we can anonymize the data to achieve an adequate functional suitability in the AI context while maintaining the privacy of the anonymized data as high as possible.

Software Engineering is a field of new challenges every day. With every passing day, new technologies emerge. There was an era of web Applications, but the time has changed and most of the web Applications are available as Mobile Applications as well. The Mobile Applications are either android based or iOS based. To deliver error free, secure and reliable Application, it is necessary to test the Applications properly. Software testing is a phase of software development life cycle, where we test an Application in all aspects. Nowadays different type of tools are available for testing an Application automatically but still we have too many challenges for applying test cases on a given Application. In this paper the authors will discuss the challenges of automation of test cases for a Mobile based payment Application.

The purpose of this paper is to analyze all Cloud based Service Models, Continuous Integration, Deployment and Delivery process and propose an Automated Continuous Testing and testing as a service based TestBot and metrics dashboard which will be integrated with all existing automation, bug logging, build management, configuration and test management tools. Recently cloud is being used by organizations to save time, money and efforts required to setup and maintain infrastructure and platform. Continuous Integration and Delivery is in practice nowadays within Agile methodology to give capability of multiple software releases on daily basis and ensuring all the development, test and Production environments could be synched up quickly. In such an agile environment there is need to ramp up testing tools and processes so that overall regression testing including functional, performance and security testing could be done along with build deployments at real time. To support this phenomenon, we researched on Continuous Testing and worked with industry professionals who are involved in architecting, developing and testing the software products. A lot of research has been done towards automating software testing so that testing of software product could be done quickly and overall testing process could be optimized. As part of this paper we have proposed ACT TestBot tool, metrics dashboard and coined 4S quality metrics term to quantify quality of the software product. ACT testbot and metrics dashboard will be integrated with Continuous Integration tools, Bug reporting tools, test management tools and Data Analytics tools to trigger automation scripts, continuously analyze application logs, open defects automatically and generate metrics reports. Defect pattern report will be created to support root cause analysis and to take preventive action.

Just like bugs in single-threaded programs can lead to vulnerabilities, bugs in multithreaded programs can also lead to concurrency attacks. We studied 31 real-world concurrency attacks, including privilege escalations, hijacking code executions, and bypassing security checks. We found that compared to concurrency bugs' traditional consequences (e.g., program crashes), concurrency attacks' consequences are often implicit, extremely hard to be observed and diagnosed by program developers. Moreover, in addition to bug-inducing inputs, extra subtle inputs are often needed to trigger the attacks. These subtle features make existing tools ineffective to detect concurrency attacks. To tackle this problem, we present OWL, the first practical tool that models general concurrency attacks' implicit consequences and automatically detects them. We implemented OWL in Linux and successfully detected five new concurrency attacks, including three confirmed and fixed by developers, and two exploited from previously known and well-studied concurrency bugs. OWL has also detected seven known concurrency attacks. Our evaluation shows that OWL eliminates 94.1% of the reports generated by existing concurrency bug detectors as false positive, greatly reducing developers' efforts on diagnosis. All OWL source code, concurrency attack exploit scripts, and results are available on github.com/hku-systems/owl.

Access Control Policies are used to specify who can access which resource under which conditions, and ensuring their correctness is vital to prevent security breaches. As access control policies can be complex and error-prone, we propose an original framework that supports the validation of the implemented policies (specified in the standard XACML notation) against the intended rights, which can be informally expressed, e.g. in tabular form. The framework relies on well-known software testing technology, such as mutation and combinatorial techniques. The paper presents the implemented environment and an application example.

Testing a software product line such as Linux implies building the source with different configurations. Manual approaches to generate configurations that enable code of interest are doomed to fail due to the high amount of variation points distributed over the feature model, the build system and the source code. Research has proposed various approaches to generate covering configurations, but the algorithms show many drawbacks related to run-time, exhaustiveness and the amount of generated configurations. Hence, analyzing an entire Linux source can yield more than 30 thousand configurations and thereby exceeds the limited budget and resources for build testing. In this paper, we present an approach to fill the gap between a systematic generation of configurations and the necessity to fully build software in order to test it. By merging previously generated configurations, we reduce the number of necessary builds and enable global variability-aware testing. We reduce the problem of merging configurations to finding maximum cliques in a graph. We evaluate the approach on the Linux kernel, compare the results to common practices in industry, and show that our implementation scales even when facing graphs with millions of edges.

Software structure analysis is crucial in software testing. Using complex network theory, we present a series of methods and build a two-layer network model for software analysis, including network metrics calculation and features extraction. Through identifying the critical functions and reused modules, we can reduce nearly 80% workload in software testing on average. Besides, the structure network shows some interesting features that can assist to understand the software more clearly.