HARSAM: A Hybrid Model for Recommendation Supported by Self-Attention Mechanism

Peng, Dunlu; Yuan, Weiwei; Liu, Cong

doi:10.1109/access.2019.2892565

Cited by 17 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yu et al [33] introduced an attention mechanism to filter user behavior sequences when modeling the short-term preferences of users and adaptively fused the long-term and short-term preferences of users based on the attention framework. Peng et al [34] proposed a hybrid model, HARSAM, which combines the attention mechanism with a deep neural network and uses the self-attention mechanism to extract the correlation between items in different time intervals. To extract more hidden information from the sparse check-in data of users, Pang et al [35] used a hierarchical attention mechanism with a "local-global" structure to mine the contribution of different features for recommendations and used this information with more contributions to make recommendations.…”

Section: Recommendations Integrating Attention Mechanismmentioning

confidence: 99%

A Graph Neural Network Social Recommendation Algorithm Integrating the Multi-Head Attention Mechanism

Liu

et al. 2023

Electronics

View full text Add to dashboard Cite

Collaborative filtering recommendation systems are facing the data sparsity problem associated with interaction data, and social recommendations introduce user social information to alleviate this problem. Existing social recommendation methods cannot express the user interaction interest and social influence deeply, which limits the recommendation performance of the system. To address this problem, in this paper we propose a graph neural network social recommendation algorithm integrating multi-head attention mechanism. First, based on the user-item interaction graph and social network graph, the graph neural network is used to learn the high-order relationship between users and items and deeply extract the latent features of users and items. In the process of learning user embedding vector representation based on the social network graph, the multi-head attention mechanism is introduced to increase the importance of friends with high influence. Then, we make rating predictions for the target users according to the learned user embedding vector representation and item embedding vector. The experimental results on the Epinions dataset show that the proposed method outperforms the existing methods in terms of both Recall and Normalized Discounted Cumulative Gain.

show abstract

Section: Recommendations Integrating Attention Mechanismmentioning

confidence: 99%

A Graph Neural Network Social Recommendation Algorithm Integrating the Multi-Head Attention Mechanism

Liu

et al. 2023

Electronics

View full text Add to dashboard Cite

show abstract

“…After applying expressions (12) and (13) in (8) and (9) we get the F-SGD weight update rules for user features and items features vectors as:…”

Section: A Fractional Sgdmentioning

confidence: 99%

“…There are different types of recommender systems based on different methods [8]- [13] such as collaborative filtering (CF), content based filtering (CB), demographic, knowledgebased, community-based and hybrid recommender systems. Widely applied techniques among those are CF [14]- [18] and CB [19]- [21].…”

Section: Introductionmentioning

confidence: 99%

Design of Momentum Fractional Stochastic Gradient Descent for Recommender Systems

Khan

Zubair²,

Alquhayz

et al. 2019

IEEE Access

View full text Add to dashboard Cite

The demand for recommender systems in E-commerce industry has increased tremendously. Efficient recommender systems are being proposed by different E-business companies with the intention to give users accurate and most relevant recommendation of products from huge amount of information. To improve the performance of recommender systems, various stochastic variants of gradient descent based algorithms have been reported. The scalability requirement of recommender systems needs algorithms with fast convergence to generate recommendations of specific items. Using the concepts of fractional calculus, an efficient variant of the stochastic gradient descent (SGD) was developed for fast convergence. Such fractional SGD (F-SGD) is further accelerated by adding a momentum term, thus termed as momentum fractional stochastic gradient descent (mF-SGD). The proposed mF-SGD method is shown to offer improved estimation accuracy and convergence rate, as compared to F-SGD and standard momentum SGD for different proportions of previous gradients, fractional orders, learning rates and number of features.

show abstract

“…With the remarkable results Transformer has achieved, there are more researches based on the self-attention mechanism. It has been used successfully in a variety of tasks to capture the global dependencies among each input sequence, such as speaker identification, relation extraction, popularity prediction, deep face recognition and recommendation systems [22]- [28]. We are the first to introduce the self-attention mechanism to the simile recognition task.…”

Section: B Self-attention Mechanismmentioning

confidence: 99%

Combining Part-of-Speech Tags and Self-Attention Mechanism for Simile Recognition

et al. 2019

View full text Add to dashboard Cite

Simile recognition is to find simile sentences and extract the tenor and vehicle from these sentences. Previous works illustrate that tenors and vehicles are typically noun phrases. A word may have different part-of-speech (POS) labels (e.g., adjectives, adverbs, nouns, and verbs) in different sentences. It is important for the simile recognition task to identify a certain POS information for each word in a sentence. However, existing models use the same word embedding to represent a word, which cannot accurately represent the POS information of this word in different sentences. In this paper, we propose a neural network framework explicitly integrating the POS information into simile recognition task, with additional self-attention mechanism to better capture long term dependencies between any two tokens in sentences. The experimental results show that our proposed models significantly outperform previous state-of-the-art methods in the simile recognition task. We also present an analysis showing that the POS information and self-attention mechanism are effective for the simile recognition task. INDEX TERMS Simile recognition, part-of-speech, self-attention mechanism.

show abstract

HARSAM: A Hybrid Model for Recommendation Supported by Self-Attention Mechanism

Cited by 17 publications

References 18 publications

A Graph Neural Network Social Recommendation Algorithm Integrating the Multi-Head Attention Mechanism

A Graph Neural Network Social Recommendation Algorithm Integrating the Multi-Head Attention Mechanism

Design of Momentum Fractional Stochastic Gradient Descent for Recommender Systems

Combining Part-of-Speech Tags and Self-Attention Mechanism for Simile Recognition

Contact Info

Product

Resources

About