Framework for Formal Verification of Machine Learning Based Complex System‐of‐Systems

Abstract

A complex system is characterized by emergence of global properties which are very difficult, if not impossible, to anticipate just from complete knowledge of component behaviors. Emergence, hierarchical organization, and numerosity are some of the characteristics of complex systems. Recently, there has been an exponential increase on the adoption of various neural network‐based machine learning models to govern the functionality and behavior of systems. With this increasing system complexity, achieving confidence in systems becomes even more difficult. Further, ease of interconnectivity among systems is permeating numerous system‐of‐systems, wherein multiple independent systems are expected to interact and collaborate to achieve unparalleled levels of functionality. Traditional verification and validation approaches are often inadequate to bring in the nuances of potential emergent behavior in a system‐of‐systems, which may be positive or negative. This paper describes a novel approach towards application of machine learning based classifiers and formal methods for analyzing and evaluating emergent behavior of complex system‐of‐systems that comprise a hybrid of constituent systems governed by conventional models and machine learning models. The proposed approach involves developing a machine learning classifier model that learns on potential negative and positive emergent behaviors, and predicts the behavior exhibited. A formal verification model is then developed to assert negative emergent behavior. The approach is illustrated through the case of a swarm of autonomous UAVs flying in a formation, and dynamically changing the shape of the formation, to support varying mission scenarios. The effectiveness and performance of the approach are quantified.

The Classification of Movement in Infants for the Autonomous Monitoring of Neurological Development

“…Additionally, if such a system is deployed in a healthcare environment, it would be of benefit to look at the transparency of the networks to better understand their decision-making process. One method of improvement would be to evaluate the formal verification of deep neural networks [ 46 , 47 ] to best ensure that their behaviour is in line with what we would expect.…”

A Machine Learning-Based Decision Support System for Predicting and Repairing Cracks in Undisturbed Loess Using Microbial Mineralization and the Internet of Things

“…In the future, we will take formal approaches into consideration for the verification of AI-based solutions [29]. In the creation and validation of AI-based solutions, formal methods, which utilize mathematical models and logic to study and verify the correctness of systems, have grown in importance [30]. Researchers may make sure that their methods are dependable, resilient, and devoid of mistakes or biases by using formal approaches.…”

AI-Enabled Consensus Algorithm in Human-Centric Collaborative Computing for Internet of Vehicle

“…In this section, based on the classical consensus algorithm, we proposed AI-PoV by adding BP neural network to the integral model and Informer prediction model to the classification model respectively [30,31]. In the context of each vehicle as a public chain composed of blockchain nodes, the higher the score, the better the reliability of the node.…”