Transformer quantum state: A multipurpose model for quantum many-body problems

Zhang, Yuan-Hang; Ventra, Massimiliano Di

doi:10.1103/physrevb.107.075147

Cited by 14 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the past years, extension of transformers to image processing (vision transformers) has proven itself to be comparably powerful to convolutions [73], opening possible routes toward their application in many-body physics [74][75][76]. The architecture of a vision transformer is schematically illustrated in Fig.…”

Section: Transformer Encodermentioning

confidence: 99%

Fluctuation based interpretable analysis scheme for quantum many-body snapshots

Schlömer,

Bohrdt

2023

SciPost Phys.

View full text Add to dashboard Cite

Microscopically understanding and classifying phases of matter is at the heart of strongly-correlated quantum physics. With quantum simulations, genuine projective measurements (snapshots) of the many-body state can be taken, which include the full information of correlations in the system. The rise of deep neural networks has made it possible to routinely solve abstract processing and classification tasks of large datasets, which can act as a guiding hand for quantum data analysis. However, though proven to be successful in differentiating between different phases of matter, conventional neural networks mostly lack interpretability on a physical footing. Here, we combine confusion learning [Nat. Phys. 13, 435 (2017)] with correlation convolutional neural networks [Nat. Commun. 12, 3905 (2021)], which yields fully interpretable phase detection in terms of correlation functions. In particular, we study thermodynamic properties of the 2D Heisenberg model, whereby the trained network is shown to pick up qualitative changes in the snapshots above and below a characteristic temperature where magnetic correlations become significantly long-range. We identify the full counting statistics of nearest neighbor spin correlations as the most important quantity for the decision process of the neural network, which go beyond averages of local observables. With access to the fluctuations of second-order correlations — which indirectly include contributions from higher order, long-range correlations — the network is able to detect changes of the specific heat and spin susceptibility, the latter being in analogy to magnetic properties of the pseudogap phase in high-temperature superconductors [Phys. Rev. Lett. 62, 957 (1989)]. By combining the confusion learning scheme with transformer neural networks, our work opens new directions in interpretable quantum image processing being sensible to long-range order.

show abstract

Section: Transformer Encodermentioning

confidence: 99%

Fluctuation based interpretable analysis scheme for quantum many-body snapshots

Schlömer,

Bohrdt

2023

SciPost Phys.

View full text Add to dashboard Cite

show abstract

“…Our approach is inspired by the success of foundation models in language 22 or vision 23 , 24 , where models are extensively pre-trained and then applied to new tasks—either without any subsequent training (referred to as zero-shot evaluation) or after small amount of training on the new task (referred to as fine-tuning). Zhang et al 25 have shown that this paradigm can be successfully applied to wavefunctions, in their case for model Hamiltonians in second quantization.…”

Section: Introductionmentioning

confidence: 99%

Towards a transferable fermionic neural wavefunction for molecules

Scherbela,

Gerard,

Grohs

2024

Nat Commun

View full text Add to dashboard Cite

Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schrödinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.

show abstract