Answer selection is a crucial subtask in the question answering (QA) system. Conventional avenues for this task mainly concentrate on developing linguistic tools that are limited in both performance and practicability. Answer selection approaches based on deep learning have been well investigated with the tremendous success of deep learning in natural language processing. However, the traditional neural networks employed in existing answer selection models, i.e., recursive neural network or convolutional neural network, typically suffer from obtaining the global text information due to their operating mechanisms. The recent Transformer neural network is considered to be good at extracting the global information by employing only self-attention mechanism. Thus, in this paper, we design a Transformer-based neural network for answer selection, where we deploy a bidirectional long short-term memory (BiLSTM) behind the Transformer to acquire both global information and sequential features in the question or answer sentence. Different from the original Transformer, our Transformer-based network focuses on sentence embedding rather than the seq2seq task. In addition, we employ a BiLSTM rather than utilizing the position encoding to incorporate sequential features as the universal Transformer does. Furthermore, we apply three aggregated strategies to generate sentence embeddings for question and answer, i.e., the weighted mean pooling, the max pooling, and the attentive pooling, leading to three corresponding Transformer-based models, i.e., QA-TF WP , QA-TF MP , and QA-TF AP , respectively. Finally, we evaluate our proposals on a popular QA dataset WikiQA. The experimental results demonstrate that our proposed Transformer-based answer selection models can produce a better performance compared with several competitive baselines. In detail, our best model outperforms the state-of-the-art baseline by up to 2.37%, 2.83%, and 3.79% in terms of MAP, MRR, and accuracy, respectively.

Document representation is widely used in practical application, for example, sentiment classification, text retrieval, and text classification. Previous work is mainly based on the statistics and the neural networks, which suffer from data sparsity and model interpretability, respectively. In this paper, we propose a general framework for document representation with a hierarchical architecture. In particular, we incorporate the hierarchical architecture into three traditional neural-network models for document representation, resulting in three hierarchical neural representation models for document classification, that is, TextHFT, TextHRNN, and TextHCNN. Our comprehensive experimental results on two public datasets, that is, Yelp 2016 and Amazon Reviews (Electronics), show that our proposals with hierarchical architecture outperform the corresponding neural-network models for document classification, resulting in a significant improvement ranging from 4.65% to 35.08% in terms of accuracy with a comparable (or substantially less) expense of time consumption. In addition, we find that the long documents benefit more from the hierarchical architecture than the short ones as the improvement in terms of accuracy on long documents is greater than that on short documents.

Session-aware recommendation is a special form of sequential recommendation, where users' previous interactions before current session are available. Recently, Recurrent Neural Network (RNN) based models are widely used in sequential recommendation tasks with great success. Previous works mainly focus on the interaction sequences of the current session without analyzing a user's long-term preferences. In this paper, we propose a joint neural network (JNN) for session-aware recommendation, which employs a Convolutional Neural Network(CNN) and a RNN to process the long-term historical interactions and the short-term sequential interactions respectively. Then, we apply a fully-connected neural network to study the complex relationship between these two types of features, which aims to generate a unified representation of the current session. Finally, a recommendation score for given items is generated by a bi-linear scheme upon the session representation. We conduct our experiments on three public datasets, showing that JNN outperforms the state-of-the-art baselines on all datasets in terms of Recall and Mean Reciprocal Rank (MRR). The outperforming results indicate that proper handling of historical interactions can improve the effectiveness of recommendation. The experimental results show that JNN is more prominent in samples with short current session or long historical interactions. INDEX TERMS Session-aware recommendation, sequential recommendation, recurrent neural networks, convolutional neural networks.

Session-aware recommendation is a special form of session-based recommendation, where users' historical interactions before the current session are available. Among the existing session-aware recommendation studies, recurrent neural network (RNN) is a popular choice to model users' current intent of the ongoing session as well as their general preference implied in previous sessions. However, these RNN-based methods present limited memory so as to have difficulty in characterizing long-term behaviour. In addition, when modeling a user's preference, existing studies mainly refer to his (her) own sessions, while ignore the collaborative information from sessions of other users that may share similar tastes. To tackle the above problems, we propose a neighbor-guided memory-based neural network (MNN) for session-aware recommendation task, which comprehensively considers users' short-term intent, long-term preference and cross-sessions information to yield the final recommendations. Specifically, we design a long-term memory generator to capture users' general preference from their historical sessions, meanwhile leverage the neighbor sessions of current session to obtain cross-session collaborative information. Furthermore, with the guidance of long-term memory and cross-session information, we employ a short-term memory generator to yield users' ongoing intent, which serves as the dominating part of recommendation. Extensive experimental results on three real world datasets show the effectiveness of our model, with improvements up to 12.0% on 30MUSIC, 26.5% on NOWPLAYING and 13.5% on TMALL in terms of Recall@20, respectively. Analysis on current session length and the number of historical interacted items comprehensively demonstrates the superiority of our proposal on processing long sessions and modeling users' long-term behaviour.

Recently, Graph Neural Networks (GNNs) have attracted increasing attention in the field of session-based recommendation, due to its strong ability on capturing complex interactive transitions within sessions. However, existing GNN-based models either lack the use of user's long-term historical behaviors or fail to address the impact of collaborative filtering information from neighbor users on the current session, which are both important to boost recommendation. In addition, previous work only focuses on the sequential relations of interactions while neglects the time interval information which can imply the correlations between different interactions. To tackle these problems, we propose a Time-Aware Graph Neural Network (TA-GNN) for session-based recommendation. Specifically, we first construct a user behavior graph by linking the interacted items of the same user according to their corresponding time order. A time-aware generator is designed to model the correlations between different nodes of the user behavior graph by considering the time interval information. Moreover, items from the neighbor sessions of the current session are selected to build a neighborhood graph. Then the two graphs are respectively processed by two different modules to learn the representation of the current session, which is applied to produce the final recommendation list. Comprehensive experiments show that our model outperforms state-of-the-art baselines on three real world datastes. We also investigate the performance of TA-GNN on different numbers of historical interactions and on different session length, finding that our model presents consistently advantages under different conditions.