Denoising Self-Attentive Sequential Recommendation

Chen, Huiyuan; Lin, Yi; Pan, Menghai; Wang, Lan; Yeh, Chin‐Chia Michael; Li, Xiaoting; Zheng, Yan; Wang, Fei; Yang, Hao

doi:10.1145/3523227.3546788

Cited by 39 publications

(15 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To optimize the GRU model, we applied a similar approach as for the LSTM model, wherein we replaced the LSTM cells in the RNN architecture with GRU cells. • Transformer (TF): The transformer is an alternative to the RNNs for sequence modeling [5,19,20,35,45]. To learn the hidden representation for the input time series, we utilized the transformer encoder proposed in [35].…”

Section: Methodsmentioning

confidence: 99%

“…Another family of methods that can be used in our problem is neural networks. For example, long short-term memory networks [15,17,18,20,45], gated recurrent unit networks [7,20,45], transformers [5,19,20,35,45], and convolutional neural networks [16,29,39] have shown effectiveness in tasks such as time series classification, forecasting, and anomaly detection. Thus, we also include these neural network models in our experiments.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Content-based Time Series Retrieval System

Yeh,

Chen,

Dai

et al. 2023

Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

A Content-based Time Series Retrieval (CTSR) system is an information retrieval system for users to interact with time series emerged from multiple domains, such as finance, healthcare, and manufacturing. For example, users seeking to learn more about the source of a time series can submit the time series as a query to the CTSR system and retrieve a list of relevant time series with associated metadata. By analyzing the retrieved metadata, users can gather more information about the source of the time series. Because the CTSR system is required to work with time series data from diverse domains, it needs a high-capacity model to effectively measure the similarity between different time series. On top of that, the model within the CTSR system has to compute the similarity scores in an efficient manner as the users interact with the system in real-time. In this paper, we propose an effective and efficient CTSR model that outperforms alternative models, while still providing reasonable inference runtimes. To demonstrate the capability of the proposed method in solving business problems, we compare it against alternative models using our in-house transaction data. Our findings reveal that the proposed model is the most suitable solution compared to others for our transaction data problem.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

An Efficient Content-based Time Series Retrieval System

Yeh,

Chen,

Dai

et al. 2023

Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the Amazon product review dataset, we followed a similar approach as described in [6,22,27]. For each user, we included the last 10 reviews in the test dataset.…”

Section: Methodsmentioning

confidence: 99%

“…TLSRec [7] simultaneously models the global stability and local fluctuation of a user's preference with a hierarchical attention network. Rec-Denosier [6] adaptively eliminates the noisy items during the training process to remove irrelevant information in a user's behavior sequence. Transformers4Rec [10] performs an empirical analysis with broad experiments of various transformer architectures for the task of sequential recommendation.…”

Section: Related Workmentioning

confidence: 99%

FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data

Zhang,

Wang,

Dai

et al. 2023

Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

Sequential tabular data is one of the most commonly used data types in real-world applications. Different from conventional tabular data, where rows in a table are independent, sequential tabular data contains rich contextual and sequential information, where some fields are dynamically changing over time and others are static. Existing transformer-based approaches analyzing sequential tabular data overlook the differences between dynamic and static fields by replicating and filling static fields into each record, and ignore temporal information between rows, which leads to three major disadvantages: (1) computational overhead, (2) artificially simplified data for masked language modeling pre-training task that may yield less meaningful representations, and (3) disregarding the temporal behavioral patterns implied by time intervals. In this work, we propose FATA-Trans, a model with two field transformers for modeling sequential tabular data, where each processes static and dynamic field information separately. FATA-Trans is fieldand time-aware for sequential tabular data. The field-type embedding in the method enables FATA-Trans to capture differences between static and dynamic fields. The time-aware position embedding exploits both order and time interval information between rows, which helps the model detect underlying temporal behavior in a sequence. Our experiments on three benchmark datasets

show abstract

“…In recent years, numerous deep learning-based recommender system models have been proposed, as documented in [21]- [24]. A particular strand of research aims to substitute the conventional inner-product operation found in Matrix Factorization (MF) models with deep neural networks.…”

Section: B Deep Learning For Collaborative Filteringmentioning

confidence: 99%

Not All Memories Created Equal: Dynamic User Representations for Collaborative Filtering

et al. 2023

View full text Add to dashboard Cite

Collaborative filtering methods for recommender systems tend to represent users as a single static latent vector. However, user behavior and interests may dynamically change in the context of the recommended item being presented to the user. For example, in the case of movie recommendations, it is usually true that movies that the user watched more recently are more informative than movies that were watched a long time ago. However, it is possible that a particular movie from the past may become suddenly more relevant for prediction in the presence of a recommendation for its sequel movie. In response to this issue, we introduce the Attentive Item2Vec++ (AI2V++) model, a neural attentive collaborative filtering approach in which the user representation adapts dynamically in the presence of the recommended item. AI2V++ employs a novel context-target attention mechanism in order to learn and capture different characteristics of the user's historical behavior with respect to a potential recommended item. Furthermore, analysis of the neural-attentive scores allows for improved interpretability and explainability of the model. We evaluate our proposed approach on five publicly available datasets and demonstrate its superior performance in comparison to state-of-the-art baselines across multiple accuracy metrics.INDEX TERMS artificial neural networks, collaborative filtering, neural attention, recommender systems.

show abstract

Denoising Self-Attentive Sequential Recommendation

Cited by 39 publications

References 29 publications

An Efficient Content-based Time Series Retrieval System

An Efficient Content-based Time Series Retrieval System

FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data

Not All Memories Created Equal: Dynamic User Representations for Collaborative Filtering

Contact Info

Product

Resources

About