A Comparison of Efficiency Improvement for Long Short-Term Memory Model Using Convolutional Operations and Convolutional Neural Network

Phankokkruad, Manop; Wacharawichanant, Sirirat

doi:10.1109/icoiact46704.2019.8938410

Cited by 5 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10. For computational efficiency, we use 𝑆𝐸 𝐿𝑈 again, instead of 𝑡𝑎𝑛ℎ as activation function [53]. Such models take about 2 weeks on a Tesla V100 GPU to reach asymptotic loss values.…”

Section: B Long Short-term Memory (Lstm)mentioning

confidence: 99%

Deep learning for surrogate modelling of 2D mantle convection

Agarwal,

Tosi,

Kessel

et al. 2021

Preprint

View full text Add to dashboard Cite

Mantle convection, the buoyancy-driven creeping flow of silicate rocks in the interior of terrestrial planets like Earth, Mars, Mercury and Venus, plays a fundamental role in the long-term thermal evolution of these bodies. Yet, key parameters and initial conditions of the partial differential equations governing mantle convection are poorly constrained. This often requires a large sampling of the parameter space to determine which combinations can satisfy certain observational constraints. Traditionally, 1D models based on scaling laws used to parameterized convective heat transfer, have been used to tackle the computational bottleneck of high-fidelity forward runs in 2D or 3D. However, these are limited in the amount of physics they can model (e.g. depth dependent material properties) and predict only mean quantities such as the mean mantle temperature. A recent machine learning study has shown that feedforward neural networks (FNN) trained using a large number of 2D simulations can overcome this limitation and reliably predict the evolution of entire 1D laterally-averaged temperature profile in time for complex models [1]. We now extend that approach to predict the full 2D temperature field, which contains more information in the form of convection structures such as hot plumes and cold downwellings. Using a dataset of 10, 525 two-dimensional simulations of the thermal evolution of the mantle of a Mars-like planet, we show that deep learning techniques can produce reliable parameterized surrogates (i.e. surrogates that predict state variables such as temperature based only on parameters) of the underlying partial differential equations. We first use convolutional autoencoders to compress the temperature fields by a factor of 142 and then use FNN and long-short term memory networks (LSTM) to predict the compressed fields. On average, the FNN predictions are 99.30% and the LSTM predictions are 99.22% accurate with respect to unseen simulations. Proper orthogonal decomposition (POD) of the LSTM and FNN predictions shows that despite a lower mean absolute relative accuracy, LSTMs capture the flow dynamics better than FNNs. When summed, the POD coefficients from FNN predictions and from LSTM predictions amount to 96.51% and 97.66% relative to the coefficients of the original simulations, respectively. I. INTRODUCTIONStudying the long-term thermal evolution of terrestrial planets like Earth, Venus, Mercury and Mars requires detailed modelling of thermal convection in their rocky mantles (e.g., [2]). Similar

show abstract

Section: B Long Short-term Memory (Lstm)mentioning

confidence: 99%

Deep learning for surrogate modelling of 2D mantle convection

Agarwal,

Tosi,

Kessel

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…10. For computational efficiency, we use SELU again, instead of tanh as activation function [64]. Such models take about 2 wk on a Tesla V100 GPU to reach asymptotic loss values.…”

Section: B Long Short-term Memory (Lstm)mentioning

confidence: 99%