A Multiscale Neural Network Based on Hierarchical Matrices

Abstract

In this work we introduce a new multiscale artificial neural network based on the structure of H-matrices. This network generalizes the latter to the nonlinear case by introducing a local deep neural network at each spatial scale. Numerical results indicate that the network is able to efficiently approximate discrete nonlinear maps obtained from discretized nonlinear partial differential equations, such as those arising from nonlinear Schrödinger equations and the Kohn-Sham density functional theory.

Effect of Mixed Precision Computing on H-Matrix Vector Multiplication in BEM Analysis

“…Finally, our study is potentially universal, and can be translated to numerous other solving techniques, including the Generalized Minimal RESidual (GMRES) method linear solver. This comes when the exploded interest in deep learning in recent years and approximating neural network model with H-matrix is taking foot [9,10].…”

Solving electrical impedance tomography with deep learning

“…Since the convolution in (3.9) is global, the architectural parameters are chosen with wN cnn ≥ N s (3.10) so that the resulting network is capable of capturing global interactions. When N s is large, it is possible that the recently proposed multiscale neural networks, for example MNN-H-net [25], MNN-H 2 -net [24], and BCR-net [23], are more efficient for such global interactions. However in order to simplify the presentation, the discussion here sticks to the convolutional layers.…”

“…the application of K T toλ can be approximated with a one-dimensional convolutional neural network, similar to K. For the part K T K + I, which can be viewed as a post-processing in the (ρ, θ) space, we implement this with several two-dimensional convolutional layers for simplicity. However, for problems with larger sizes, multiscale neural networks such as [25,24,23] can be also used. The resulting architecture for the inverse map is summarized in Algorithm 2 and illustrated in Section 3.2…”

“…In the recent years, deep neural networks (DNNs) have been very effective tools in a variety of contexts and have achieved great successes in computer vision, image processing, speech recognition, and many other artificial intelligence applications [39,46,33,54,50,60,48,58]. More recently, DNNs have been increasingly used in the context of scientific computing, particularly in solving PDE-related problems [43,8,35,25,3,55,47,28]. First, since neural networks offer a powerful tool for approximating high-dimensional functions [17], it is natural to use them as an ansatz for high-dimensional PDEs [57,11,35,44,20].…”

“…First, since neural networks offer a powerful tool for approximating high-dimensional functions [17], it is natural to use them as an ansatz for high-dimensional PDEs [57,11,35,44,20]. A second main direction focuses on the low-dimensional parameterized PDE problems, by using the DNNs to represent the nonlinear map from the high-dimensional parameters of the PDE solution [52,36,43,25,24,23,51,7]. Applying DNNs to inverse problems [45,40,41,2,53,61,26,56] can be viewed as a particularly important case of this direction.…”