Evolutional deep neural network

Du, Yifan; Zaki, Tamer A.

doi:10.1103/physreve.104.045303

Cited by 43 publications

(29 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, it should also be emphasized that we can introduce a loss function associated with a priori knowledge from physics since both NNs and LSE are based on a minimization manner in terms of weights. Inserting a physical loss function may be one of the considerable paths towards practical applications of both methods 34 – 36 .…”

Section: Discussionmentioning

confidence: 99%

Identifying key differences between linear stochastic estimation and neural networks for fluid flow regressions

Nakamura

Fukami

Fukagata

2022

Sci Rep

View full text Add to dashboard Cite

Neural networks (NNs) and linear stochastic estimation (LSE) have widely been utilized as powerful tools for fluid-flow regressions. We investigate fundamental differences between them considering two canonical fluid-flow problems: (1) the estimation of high-order proper orthogonal decomposition coefficients from low-order their counterparts for a flow around a two-dimensional cylinder, and (2) the state estimation from wall characteristics in a turbulent channel flow. In the first problem, we compare the performance of LSE to that of a multi-layer perceptron (MLP). With the channel flow example, we capitalize on a convolutional neural network (CNN) as a nonlinear model which can handle high-dimensional fluid flows. For both cases, the nonlinear NNs outperform the linear methods thanks to nonlinear activation functions. We also perform error-curve analyses regarding the estimation error and the response of weights inside models. Our analysis visualizes the robustness against noisy perturbation on the error-curve domain while revealing the fundamental difference of the covered tools for fluid-flow regressions.

show abstract

Section: Discussionmentioning

confidence: 99%

Identifying key differences between linear stochastic estimation and neural networks for fluid flow regressions

Nakamura

Fukami

Fukagata

2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Closest to our model is the work in [6], which also proposes to integrate the DNN parametrizations of PDE solutions sequentially in time. However, Ref.…”

Section: Related Workmentioning

confidence: 99%

“…In this work we take a different approach. We develop time-integrators for PDEs that use DNNs to represent the solution but update the parameters sequentially from one time slice to another rather than globally over the whole time-space domain, as was also proposed in [6], thereby allowing for collecting data on the way and for integration over arbitrary long time intervals. The scheme uses the structural form of the PDEs, but no a priori data about their solution.…”

Section: Introductionmentioning

confidence: 99%

Neural Galerkin Scheme with Active Learning for High-Dimensional Evolution Equations

Bruna¹,

Peherstorfer²,

Vanden‐Eijnden³

2022

Preprint

View full text Add to dashboard Cite

Machine learning methods have been shown to give accurate predictions in high dimensions provided that sufficient training data are available. Yet, many interesting questions in science and engineering involve situations where initially no data are available and the principal aim is to gather insights from a known model. Here we consider this problem in the context of systems whose evolution can be described by partial differential equations (PDEs). We use deep learning to solve these equations by generating data on-the-fly when and where they are needed, without prior information about the solution. The proposed Neural Galerkin schemes derive nonlinear dynamical equations for the network weights by minimization of the residual of the time derivative of the solution, and solve these equations using standard integrators for initial value problems. The sequential learning of the weights over time allows for adaptive collection of new input data for residual estimation. This step uses importance sampling informed by the current state of the solution, in contrast with other machine learning methods for PDEs that optimize the network parameters globally in time. This active form of data acquisition is essential to enable the approximation power of the neural networks and to break the curse of dimensionality faced by non-adaptative learning strategies. The applicability of the method is illustrated on several numerical examples involving high-dimensional PDEs, including advection equations with many variables, as well as Fokker-Planck equations for systems with several interacting particles.

show abstract

“…In this case, we learn to represent data at t 0 with parameters θ(t 0 ), and then demand that each point of time the distribution satisfies differential constraints for a PDE in question. This leads to model-dependent updates of variational parameters θ(t + ∆t) γ ← − θ(t) (with an update rule γ), thus evolving the model in discrete time [64]. Below, we show how to introduce model-dependent differential constraints, and training or evolving DQGM in both explicit and implicit manner.…”

Section: Model Differentiation and Constrained Training From Stochast...mentioning

confidence: 99%

“…Alternatively, we can use an evolutionary approach for updating circuit parameters [64]. In this case, the time-derivative of our model ∂ p θ,t (x)/∂t can be reexpressed using a chain rule as (∂ p θ,t (x)/∂θ)(∂θ/∂t).…”

Section: Model Differentiation and Constrained Training From Stochast...mentioning

confidence: 99%

Protocols for Trainable and Differentiable Quantum Generative Modelling

Kyriienko¹,

Paine²,

Elfving³

2022

Preprint

View full text Add to dashboard Cite

We propose an approach for learning probability distributions as differentiable quantum circuits (DQC) that enable efficient quantum generative modelling (QGM) and synthetic data generation. Contrary to existing QGM approaches, we perform training of a DQCbased model, where data is encoded in a latent space with a phase feature map, followed by a variational quantum circuit. We then map the trained model to the bit basis using a fixed unitary transformation, coinciding with a quantum Fourier transform circuit in the simplest case. This allows fast sampling from parametrized distributions using a single-shot readout. Importantly, latent space training provides models that are automatically differentiable, and we show how samples from solutions of stochastic differential equations (SDEs) can be accessed by solving stationary and time-dependent Fokker-Planck equations with a quantum protocol. Finally, our approach opens a route to multidimensional generative modelling with qubit registers explicitly correlated via a (fixed) entangling layer. In this case quantum computers can offer advantage as efficient samplers, which perform complex inverse transform sampling enabled by the fundamental laws of quantum mechanics. On a technical side the advances are multiple, as we introduce the phase feature map, analyze its properties, and develop frequency-taming techniques that include qubit-wise training and feature map sparsification.

show abstract

Evolutional deep neural network

Cited by 43 publications

References 26 publications

Identifying key differences between linear stochastic estimation and neural networks for fluid flow regressions

Identifying key differences between linear stochastic estimation and neural networks for fluid flow regressions

Neural Galerkin Scheme with Active Learning for High-Dimensional Evolution Equations

Protocols for Trainable and Differentiable Quantum Generative Modelling

Contact Info

Product

Resources

About