Training neural networks using Metropolis Monte Carlo and an adaptive variant

Whitelam, Stephen; Selin, Viktor; Benlolo, Ian; Casert, Corneel; Tamblyn, Isaac

doi:10.1088/2632-2153/aca6cd

Cited by 7 publications

(5 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, compared to physically motivated ansatzes such as ADAPT that suffer significantly less from the barren plateau problem, QCMPS shows even better optimization performance for some systems (Table S5). Several techniques, such as DMRG-sweep-like method, identity block initialization, layer-wise optimization, and Monte-Carlo-based algorithm, may be able to alleviate the optimization problem. However, a detailed investigation in this direction is out of the scope of this study.…”

Section: Numerical Resultsmentioning

confidence: 99%

Quantum Circuit Matrix Product State Ansatz for Large-Scale Simulations of Molecules

Fan,

Liu,

et al. 2023

J. Chem. Theory Comput.

View full text Add to dashboard Cite

As demonstrated in the density matrix renormalization group (DMRG) method, approximating many-body wave function of electrons using a matrix product state (MPS) is a promising way to solve electronic structure problems. The expressibility of an MPS is determined by the size of the matrices or, in other words, the bond dimension, which unfortunately may be required to be very large in quantum chemistry simulations. In this study, we propose to calculate the ground state energies of molecular systems by variationally optimizing quantum circuit MPS (QCMPS) with a relatively small number of qubits. It is demonstrated that with carefully chosen circuit structure and orbital localization scheme, QCMPS can reach a similar accuracy as that achieved in DMRG with an exponentially large bond dimension. QCMPS simulation of a linear hydrogen molecular chain with 50 orbitals can reach the chemical accuracy using only 6 qubits at a moderate circuit depth. These results suggest that QCMPS is a promising wave function ansatz in the variational quantum eigensolver algorithm for molecular systems.

show abstract

Section: Numerical Resultsmentioning

confidence: 99%

Quantum Circuit Matrix Product State Ansatz for Large-Scale Simulations of Molecules

Fan,

Liu,

et al. 2023

J. Chem. Theory Comput.

View full text Add to dashboard Cite

show abstract

“…In the field of statistical physics, we usually rely on Monte Carlo algorithms for optimization, like simulated annealing [31]. Some studies have been reported on the line [32,33]. However, it is not so easy and requires some modifications [33].…”

Section: Discussionmentioning

confidence: 99%

“…Some studies have been reported on the line [32,33]. However, it is not so easy and requires some modifications [33]. Specifically, it fails when the network is highly heterogeneous.…”

Section: Discussionmentioning

confidence: 99%

A simple theory for training response of deep neural networks

Nakazato

2024

Phys. Scr.

View full text Add to dashboard Cite

Deep neural networks give us a powerful method to model the training dataset’s relationship between input and output. We can regard that as a complex adaptive system consisting of many artificial neurons that work as an adaptive memory as a whole. The network’s behavior is training dynamics with a feedback loop from the evaluation of the loss function. We already know the training response can be constant or shows power law-like aging in some ideal situations. However, we still have gaps between those findings and other complex phenomena, like network fragility. To fill the gap, we introduce a very simple network and analyze it. We show the training response consists of some different factors based on training stages, activation functions, or training methods. In addition, we show feature space reduction as an effect of stochastic training dynamics, which can result in network fragility. Finally, we discuss some complex phenomena of deep networks.

show abstract

“…In the field of statistical physics, we usually rely on Monte Carlo algorithms for optimization, like simulated annealing [30]. Some studies have been reported on the line [31,32]. However, it is not so easy and requires some modifications [32].…”

Section: Discussionmentioning

confidence: 99%

“…Some studies have been reported on the line [31,32]. However, it is not so easy and requires some modifications [32]. Specifically, it fails when the network is highly heterogeneous.…”

Section: Discussionmentioning

confidence: 99%

The training response law explains how deep neural networks learn

Nakazato

2022

J. Phys. Complex.

View full text Add to dashboard Cite

Deep neural network is the widely applied technology in this decade. In spite of the fruitful applications, the mechanism behind that is still to be elucidated. We study the learning process with a very simple supervised learning encoding problem. As a result, we found a simple law, in the training response, which describes neural tangent kernel. The response consists of a power law like decay multiplied by a simple response kernel. We can construct a simple mean-ﬁeld dynamical model with the law, which explains how the network learns. In the learning, the input space is split into sub-spaces along competition between the kernels. With the iterated splits and the aging, the network gets more complexity, but ﬁnally loses its plasticity.

show abstract

Training neural networks using Metropolis Monte Carlo and an adaptive variant

Cited by 7 publications

References 44 publications

Quantum Circuit Matrix Product State Ansatz for Large-Scale Simulations of Molecules

Quantum Circuit Matrix Product State Ansatz for Large-Scale Simulations of Molecules

A simple theory for training response of deep neural networks

The training response law explains how deep neural networks learn

Contact Info

Product

Resources

About