Neural Networks and Imbalanced Learning for Data-Driven Scientific Computing With Uncertainties

Abstract

Uncertainty quantification in complex engineering problems is challenging because of necessitating large numbers of expensive model evaluations. This paper proposes a two-stage framework for developing accurate machine learning-based surrogate models in structural engineering. The studied numerical model considers aleatory and epistemic uncertainties, i.e., ground motion features and material properties. Our framework's first step trains classification algorithms on the collected data from our numerical model with a disproportionate ratio of observations from two categories, i.e., failed and safe simulations. We investigate the performance of imbalanced learning strategies along with artificial neural networks to achieve high classification accuracy. The second step of our framework aims to estimate three quantities of interest using the same network architecture, comparing our approach with regularized linear regression models. Moreover, we present a new approach to reducing the number of numerical simulations for developing machine learning-based surrogate models with limited training data. This approach employs Gaussian processes as a powerful probabilistic technique, providing an inherent uncertainty measure to determine the quality of estimated response values. Extensive numerical experiments demonstrate the superior performance of neural networks with three hidden layers compared to traditional machine learning algorithms for both classification and regression tasks. Also, empirical investigations corroborate that Gaussian processes enable us to predict the values of missing simulations for reducing the computational cost associated with numerical models. To conclude this work, we present several applications and future research directions. INDEX TERMS Uncertainty, Gaussian processes, neural networks, imbalanced classification, regression.

Application of machine learning in polymer additive manufacturing: A review

“…ANN models are capable of capturing highly complex and non-linear relationships between inputs and outputs. 144,145 The models are inspired by the structure and functionality of the human brain. ANN models generally consist of three types of layers: an input layer, one or more hidden layers, and an output layer.…”

The automated collapse data constructor technique and the data‐driven methodology for seismic collapse risk assessment

“…Given that there are 112 elements in the feature vector as described in the next paragraph, we use overcomplete layers (i.e., larger than the input size) with all layers of the same size following the recommendations by Bengio. 58 The rationale for using two densely connected layers is based on previous studies 28,59 which found using one to three hidden layers to be effective. Note again that the objective herein is to have a surrogate architecture that is good enough to demonstrate the benefits of the 'data-centric' approach rather than focusing on optimization of hyperparameters (i.e., the 'model-centric' approach).…”

“…7-18, single-degree-of-freedom structures on liquefiable sand deposit, 19 reinforced concrete (RC) shear walls, 20 risk modeling of regional portfolios of structures, 14,15,18,[21][22][23][24] and estimation of collapse vulnerability of buildings. [25][26][27][28] However, similar to the general trends in the ML community, 29 the majority of the past research efforts focused on algorithmic developments or contrasting of the performance of different ML tools, while putting less emphasis on the most effective use of data. As an example of the latter, studies [30][31][32] focused on the use of stochastic surrogate models to better utilize results of linear and nonlinear response history analyses for risk assessments.…”

“…Previous research on application of ML tools for seismic fragility assessment revolved around development of probabilistic seismic demand models and parameterized fragility functions with application to highway bridges e.g. 7–18, single‐degree‐of‐freedom structures on liquefiable sand deposit, 19 reinforced concrete (RC) shear walls, 20 risk modeling of regional portfolios of structures, 14,15,18,21–24 and estimation of collapse vulnerability of buildings 25–28 . However, similar to the general trends in the ML community, 29 the majority of the past research efforts focused on algorithmic developments or contrasting of the performance of different ML tools, while putting less emphasis on the most effective use of data.…”

A review of the application of the simulated annealing algorithm in structural health monitoring (1995-2021)

“…To detect damages, predict seismic responses, and optimize sensor locations, the FEMs of some famous structures such as Guangzhou New TV Tower [37], Shanghai Tower [36,38], MIT Green Building [39,40], and Dalian World Trade Building [41] were simplified. Pourkamali-Anaraki and Hariri-Ardebili have presented a two-step uncertainty quantification method that uses a simplified alternative model of Milad Tower [42]. The classification of vibration-based damage detection methods is illustrated in Fig.…”