Meta-learning pseudo-differential operators with deep neural networks

Abstract

This paper introduces a meta-learning approach for parameterized pseudo-differential operators with deep neural networks. With the help of the nonstandard wavelet form, the pseudo-differential operators can be approximated in a compressed form with a collection of vectors. The nonlinear map from the parameter to this collection of vectors and the wavelet transform are learned together from a small number of matrix-vector multiplications of the pseudo-differential operator. Numerical results for Green's functions of elliptic partial differential equations and the radiative transfer equations demonstrate the efficiency and accuracy of the proposed approach.Keywords: Deep neural networks; Convolutional neural networks; Nonstandard wavelet form; Metalearning; Green's functions; Radiative transfer equation.when the operator G η is a pseudo-differential operator (PDO) [57]. Although L η and G η can be linear operators, this map M from η to G η is highly nonlinear.Background. In the recent years, deep learning has become the most versatile and effective tool in artificial intelligence and machine learning, witnessed by impressive achievements in computer vision [35,58,28], speech and natural language processing [29,53,48,54,11], drug discovery [41] or game playing [51,15,55]. Recent reviews on deep learning and its impacts on other fields can be found in for example [37,50]. At the center of deep learning, the model of deep neural networks (NNs) provides a flexible framework for approximating high-dimensional functions, while allowing for efficient training and good generalization properties in practice [40,44].More recently, several groups have started applying NNs to partial differential equations (PDEs) and integral equations (IEs) arising from physical systems. In one direction, the NN model has been used to

Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks

“…Based on H-matrix and H 2 -matrix structure, Fan and his coauthors proposed two multiscale neural networks [20,21], which are more suitable in training smooth linear or nonlinear operators due to the multiscale nature of H-matrices. In addition to that, nonstandard wavelet form inspired the design of BCR-Net [22], which is applied to address the inverse of elliptic operator and nonlinear homogenization problem and recently been embedded in a neural network for solving electrical impedance tomography [19] and pseudo-differential operator [23]. Multigrid method also inspired MgNet [25].…”

Solving electrical impedance tomography with deep learning

“…In the recent years, deep neural networks (DNNs) have been very effective tools in a variety of contexts and have achieved great successes in computer vision, image processing, speech recognition, and many other artificial intelligence applications [39,46,33,54,50,60,48,58]. More recently, DNNs have been increasingly used in the context of scientific computing, particularly in solving PDE-related problems [43,8,35,25,3,55,47,28]. First, since neural networks offer a powerful tool for approximating high-dimensional functions [17], it is natural to use them as an ansatz for high-dimensional PDEs [57,11,35,44,20].…”

Learning Elliptic Partial Differential Equations with Randomized Linear Algebra

“…If so, how many pairs are required? These two questions have received significant research attention [17,31,34,43]. From data, one hopes to eventually learn physical laws of nature or conservation laws that elude scientists in the biological sciences [63], computational fluid dynamics [49], and computational physics [45].…”

“…The approach that dominates the PDE learning literature is to directly learn L by either (1) learning parameters in the PDE [4,64], (2) using neural networks to approximate the action of the PDE on functions [45][46][47][48][49], or (3) deriving a model by composing a library of operators with sparsity considerations [9,35,52,53,59,60]. Instead of trying to learn the unbounded, closed operator L directly, we follow [6,17,18] and discover the Green's function associated with L. That is, we attempt to learn the function G : D × D → R + ∪ {∞} such that [16] u j (x) = D G(x, y) f j (y)dy,…”