No abstract

Artificial neural network, consisting of many neurons in different layers, is an important method to simulate humain brain. Usually, one neuron has two operations: one is linear, the other is nonlinear. The linear operation is inner product and the nonlinear operation is represented by an activation function. In this work, we introduce a kind of quantum neuron whose inputs and outputs are quantum states. The inner product and activation operator of the quantum neurons can be realized by quantum circuits. Based on the quantum neuron, we propose a model of quantum neural network in which the weights between neurons are all quantum states. We also construct a quantum circuit to realize this quantum neural network model. A learning algorithm is proposed meanwhile. We show the validity of learning algorithm theoretically and demonstrate the potential of the quantum neural network numerically.

No abstract

With the development of feed-forward models, the default model for sequence modeling has gradually evolved to replace recurrent networks. Many powerful feed-forward models based on convolutional networks and attention mechanisms were proposed and show more potential to handle sequence modeling tasks. We wonder that is there an architecture that can not only achieve an approximate substitution of recurrent networks but also absorb the advantages of feed-forward models. So we propose an exploratory architecture referred to Temporal Convolutional Attention-based Network (TCAN) which combines temporal convolutional network and attention mechanism. TCAN includes two parts, one is Temporal Attention (TA) which captures relevant features inside the sequence, the other is Enhanced Residual (ER) which extracts the shallow layer's important information and transfers to deep layers. We improve the state-of-theart results of bpc/perplexity to 26.92 on word-level PTB, 1.043 on character-level PTB, and 6.66 on WikiText-2.