Deep tensor networks with matrix product operators

Žunkovič, Bojan

doi:10.1007/s42484-022-00081-1

Cited by 2 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A successful example is the introduction of tensor-network state into deep learning. It stems from quantum information and develops fastly in quantum many-body physics, and recently it has been used to realize the supervised learning [38][39][40], generative models [41][42][43], and network reconstruction [44][45][46], etc. Though there are some limitations and difficulties in the current stage, it can still be expected that the interplay between deep learning and many-body physics will continue to flourish in the next decade.…”

Section: Introductionmentioning

confidence: 99%

Exploring explicit coarse-grained structure in artificial neural networks

Xi-Ci¹,

Xie²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multilevel abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed.

show abstract

Section: Introductionmentioning

confidence: 99%

Exploring explicit coarse-grained structure in artificial neural networks

Xi-Ci¹,

Xie²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

Exploring Explicit Coarse-Grained Structure in Artificial Neural Networks

Xi-Ci

Xie

Yang

2023

Chinese Phys. Lett.

View full text Add to dashboard Cite

We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multi-level abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed.

show abstract

Deep tensor networks with matrix product operators

Cited by 2 publications

References 25 publications

Exploring explicit coarse-grained structure in artificial neural networks

Exploring explicit coarse-grained structure in artificial neural networks

Exploring Explicit Coarse-Grained Structure in Artificial Neural Networks

Contact Info

Product

Resources

About