Comparing Implementations of Cellular Automata as Images: A Novel Approach to Verification by Combining Image Processing and Machine Learning

Abstract

Discrete models such as cellular automata may be ported from one platform or language onto another to improve performances, for instance by rewriting legacy Matlab code into C++ or adding optimizations into a Python implementation. Although such transformations can offer benefits such as scalability or maintainability, they also have the risk of introducing bugs. While standard verification techniques can always be applied, this situation presents a unique opportunity since the two implementations can be directly compared based on their simulation runs. Although comparing average results across runs of a same configuration is a common practice, our paper shows that many bugs would not be detected at this aggregate level. We thus propose comparing implementations of cellular automata by analyzing their outputs as images. In this paper, we examine the detection of several implementation errors using five different techniques (supervised/unsupervised image processing, decision trees, random forests, or deep learning) across three different cellular automata models (forest fire, tumor, HIV). We show that in some models, random forests can detect 4 out of 5 erroneous runs, although the accuracy depends both on the model and on the nature of the errors.

Experimental evaluation of a machine learning approach to improve the reproducibility of network simulations

“…A comprehensive review of all such methods is beyond the scope of this article; hence, we briefly describe two methods that have yielded strong performances in our past studies on simulation verification. 16,17 For additional information, we refer the reader to the introductory hands-on tutorial by Ta and colleagues, 35 or dedicated volumes on classification methods either from a computer vision perspective 36 or with an artificial intelligence (AI) focus. 37 A decision tree functions by continuously splitting the feature space with axis parallel cuts to reduce the entropy of the overall system.…”

“…15 However, previous work has shown that such approach to verification can fail since aggregated totals may appear correct even if the simulation is actually buggy. 16 Hence, a thorough verification should not only check the total counts of each state but also where such states are located within the simulated population. To check such low-level patterns , we previously proposed an imagification process that converts the simulated network at each time step into an image that can then be analyzed with the use of machine learning.…”

An Empirical Study on the Training Characteristics of Weekly Load of Calisthenics Teaching and Training Based on Deep Learning Algorithm

“…In Wozniak and Giabbanelli's research on forest data recognition, the way to improve the accuracy of recognition and classification by BP neural network is discussed. e clustering algorithm is introduced to optimize the BP neural network, which makes up for the slow convergence of the BP neural network algorithm [2]. Tian et al introduced an optimized BP neural network to solve the reliability problem of excessive noise data by using a new executive [3].…”