Matrix product operators for sequence-to-sequence learning

Abstract

The method of choice to study one-dimensional strongly interacting many body quantum systems is based on matrix product states and operators. Such method allows to explore the most relevant, and numerically manageable, portion of an exponentially large space. It also allows to describe accurately correlations between distant parts of a system, an important ingredient to account for the context in machine learning tasks. Here we introduce a machine learning model in which matrix product operators are trained to implement sequence to sequence prediction, i.e. given a sequence at a time step, it allows one to predict the next sequence. We then apply our algorithm to cellular automata (for which we show exact analytical solutions in terms of matrix product operators), and to nonlinear coupled maps. We show advantages of the proposed algorithm when compared to conditional random fields and bidirectional long short-term memory neural network. To highlight the flexibility of the algorithm, we also show that it can readily perform classification tasks.The last few years have witnessed a great shift of interest of society (individuals, industries and governmental organizations) towards machine learning and its wide range of applications ranging from images classification to translation and more.In recent years we have also witnessed an increasing activity at the intersection between machine learning and quantum physics. This includes further studies on quantum machine learning [1-3], use of machine learning for materials study [4], glassy physics [5], for phases recognition [6][7][8][9][10][11][12][13] and to numerically study quantum systems [14][15][16][17][18][19][20][21][22][23]. To be noted are studies on physical analogies and reasons for effectiveness of machine learning [24][25][26].Another research direction has been to apply tools developed in many body quantum physics to typical machine learning tasks. Recent examples are [27][28][29], where algorithms based on matrix product states (MPS), also known as tensor trains, were successfully used for supervised classification or unsupervised generative modelling. MPS based algorithms were also used for classification [30], as predictive modeling of stochastic processes [31], for language modeling [32] and compared to Boltzmann machines [33,34]. Usefuleness of tensor representations and matrix product states was also noted in [35].In physics, MPS are used to represent wave-functions, probability distributions or density matrices as a product of tensors [36][37][38][39][40]. Building on the density matrix renormalization group [41], matrix product states are very succesfully used in many body quantum physics to study ground or steady states and time evolution of Hamiltonian or dissipative systems [42][43][44][45][46][47][48]. As an extension, quantum mechanical operators are represented by matrix product operators (MPO), which are composed of tensors and map an MPS to another one. One important aspect of matrix product states is that they can allow to give an accurate ...

Matrix Product States for Quantum Stochastic Modeling

“…These results thus provide a direct means of using MPS methods to study the resource requirements of quantum stochastic simulation, extending their relevance to the field of predictive modeling. Our approach complements other uses of tensor network methods for the description of classical systems with stochastic elements [35][36][37][38][39][40][41] and machine learning [42][43][44][45][46][47][48][49][50]. We extend these results in introducing causal structure, adapting MPS methods for predictive modeling.Predictive models.-Consider a system that generates an output x t sampled from a random variable X t at each time t.…”

Tensor network compressed sensing with unsupervised machine learning

“…[41]). As shown in the previous works [15,25,26,[28][29][30][31][32], TN models (including MPS) possess remarkable generalization power that is competitive to neural networks. Notably, TN models surpass neural networks as they possess high interpretability and allow us to implement quantum processes.…”

“…Recently, TN [20][21][22][23][24] is rapidly developed into a powerful quantum-inspired computational tool for machine learning, which brings new possibilities and wide perspectives to process real-life data, such as images and texts, in the quantum processes based on many-qubit (or many-body) states representing the probability distribution of the data that Alice considers to send, (2) encode the specific piece of information to be sent by projecting | , (3) decode the information as a generative process by the projected Born machine. [15,[25][26][27][28][29][30][31][32]. High efficiencies have been demonstrated at least for the classical simulations of these quantum processes.…”

Machine learning for quantum matter

“…Tensor networks have been successfully applied to several learning tasks including dimensionality reduction [250], unsupervised learning and generative modelling using matrix product states [251][252][253], representation learning with multi-scale tensor networks [254], sequence-to-sequence learning using matrix product operators [255], language modelling [256,257], Bayesian inference [258]. Ref.…”