Monotonic Neural Network for Ground-Motion Predictions to Avoid Overfitting to Recorded Sites

Abstract

Data-driven machine-learning approaches are being increasingly applied to construct empirical ground-motion models (GMMs). It is a standard practice to divide observational records into learning and test datasets to correctly evaluate the predictive performance of a developed model. However, in this study, we show that division based on records or earthquakes is inappropriate for evaluating the generalization performance on recorded sites when GMMs include site-condition proxies as input variables. Complex models exhibit small residuals at sites used in the training process, but exhibit large residuals at new sites owing to overfitting to the trained sites. As a simple solution, we propose a neural network model that has monotonic dependence on some of the input variables. The model successfully obtains the generalization performance on recorded sites, although it lacks ability to represent oversaturation with input variables suggested in extreme ground-motion ranges. Therefore, alternative methods should be investigated to develop robust data-driven models under general conditions. Dividing the sites into learning and test data would play a fundamental role in developing such robust models.

Ground motion models for regions with limited data: Data‐driven approach

“…The former approach is implemented in the current work, and the weights are constrained such that the output behaves monotonically, either non‐decreasing or non‐increasing, with inputs. Furthermore, the monotonic network reduces overfit and allows output saturation 27 ; however, it does not allow oversaturation (trend reversal). Monotonic constraint would be of little significance for regions with large amounts of data covering all possible cases, as the model would learn appropriate constraints from the data.…”

“…A monotonically increasing function such as tanh or elu 47 is chosen as an activation function. If all the weights connected to j th input in the first layer are positive and the weights in the remaining layers are positive would enforce monotonically increasing constraints on spectrum with respect to input $${{\bf x}}_{\mathrm{j}}^0$$ 27,46 . On the other hand, if all the weights connected to j th input in the first layer are negative and the weights in the remaining layers are positive would enforce the monotonic decreasing constraint on spectrum with respect to input $${{\bf x}}_{\mathrm{j}}^0$$ 27,46 These conditions are summed up in Equation ().…”

“…The transfer learning technique ensures that appropriate trends learned in the pre-trained model using the global data are transferred to the limited data model. Further, the target values are monotonically constrained with respect to inputs during training 27,28 to ensure that appropriate physics is learned.…”

A data-driven artificial neural network model for the prediction of ground motion from induced seismicity: The case of The Geysers geothermal field

“…Recent works based on such an approach have been used with the seismic data in the earthquake phenomenology (Seydoux et al, 2020;Kuang et al, 2021). Among the available data-driven models (e.g., machine learning algorithms, fuzzy logic, and Gaussian regression) we selected the artificial neural network (ANN) model (e.g., Derras et al, 2014;Kubo et al, 2020;Okazaki et al, 2021). The ANN models are built from the composition of a fixed number of aggregation operations and activation functions and provide strong flexibility in terms of predictability power.…”

“…Theoretically, there are universal approximation theorems, which guarantee the existence of ANNs having an arbitrarily small error (Cybenko, 1989). Another advantage of ANNs is that it requires no constraints in how the features in the data are distributed, contrary to the other statistical-based approaches (Derras et al, 2014;Khosravikia et al, 2019;Kubo et al, 2020;Okazaki et al, 2021). Even though they are considered "black box" and prone to overfitting (Loyola-González, 2019), recent advancements in artificial intelligence (AI), and in particular machine learning (ML) and deep learning, provide new tools to improve both the generalization and the expandability of such models, making them more reliable for real-world applications (Arrieta et al, 2019;Ahmed et al, 2022;Velasco Herrera et al, 2022).…”