CAREER: Decision Procedures for High-Assurance, AI-Controlled, Cyber-Physical Systems

Abstract: This project explores new mathematical techniques that provide a scientific basis to understand the fundamental properties of Cyber-Physical Systems (CPS) controlled by Artificial Intelligence (AI) and guide their design. From simple logical constructs to complex deep neural network models, AI agents are increasingly controlling physical/mechanical systems. Self-driving cars, drones, and smart cities are just examples of AI-controlled CPS. However, regardless of the explosion in the use of AI within a multitude of CPS domains, the safety and reliability of these AI-controlled CPS is still an under-studied problem. This project includes activities integrated with education, so as to explore how learning through counterexamples works for AI, and to help with critical thinking skills for young students. This project investigates a new generation of formal method tools that are capable of simultaneously analyzing the cyber components (including AI-agents) and the physical components of CPS. This new generation of formal methods will be used to analyze the safety and reliability of AI-controlled CPS, characterize the environments for which the system is guaranteed to operate correctly and predict their failure at real-time. In addition, this project will address the problem of how to assign the blame of system failures in AI-controlled CPS. The proposed methods will be evaluated over two flagship testbeds (i) autonomous drones and (ii) self-driving cars. The project engages graduate, undergraduate, and high-school students, and reaches out to the scientific community by providing open source implementations of algorithms.