Deep Multifaceted Transformers for Multi-objective Ranking in Large-Scale E-commerce Recommender Systems

Gu, Yun; Ding, Zhuoye; Wang, Shuaiqiang; Zou, Lixin; Liu, Yiding; Yin, Dawei

doi:10.1145/3340531.3412697

Cited by 70 publications

(39 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Multi-task learning has raised attention recently in recommendation systems [10,35]. However, the focus of existing literature is in the phase of predicting immediate user actions [8,12,15,30] and not the strategic parameters. Different architecture for model-parameter sharing, loss sharing, etc., have been explored [8,22,30,36].…”

Section: Related Workmentioning

confidence: 99%

PASTO: Strategic Parameter Optimization in Recommendation Systems -- Probabilistic is Better than Deterministic

Ding,

Tang,

Feng

et al. 2021

Preprint

View full text Add to dashboard Cite

Real-world recommendation systems often consist of two phases. In the first phase, multiple predictive models produce the probability of different immediate user actions. In the second phase, these predictions are aggregated according to a set of 'strategic parameters' to meet a diverse set of business goals, such as longer user engagement, higher revenue potential, or more community/network interactions. In addition to building accurate predictive models, it is also crucial to optimize this set of 'strategic parameters' so that primary goals are optimized while secondary guardrails are not hurt. In this setting with multiple and constrained goals, this paper discovers that a probabilistic strategic parameter regime can achieve better value compared to the standard regime of finding a single deterministic parameter. The new probabilistic regime is to learn the best distribution over strategic parameter choices and sample one strategic parameter from the distribution when each user visits the platform. To pursue the optimal probabilistic solution, we formulate the problem into a stochastic compositional optimization problem, in which the unbiased stochastic gradient is unavailable. Our approach is applied in a popular social network platform with hundreds of millions of daily users and achieves +0.22% lift of user engagement in a recommendation task and +1.7% lift in revenue in an advertising optimization scenario comparing to using the best deterministic parameter strategy.

show abstract

Section: Related Workmentioning

confidence: 99%

PASTO: Strategic Parameter Optimization in Recommendation Systems -- Probabilistic is Better than Deterministic

Ding,

Tang,

Feng

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…• DMT: DMT [10] uses multiple Transformers to model users' multiple types of behavior sequences, including click sequence of items, cart sequence of items, and order sequence of items.…”

Section: Comparedmentioning

confidence: 99%

“…Recently, researchers point out that the modeling of click sequence can only focus on what users are interested in, but ignore the modeling of what users are not interested in, which leads to the captured user preferences are biased [42]. In view of this, the interaction data is subdivided into implicit and explicit feedback [10,42]. Among them, explicit feedback is defined as precise but relatively rare feedback that can directly indicate users' positive/negative preferences in the view page, such as rating and tagging like/dislike, while implicit feedback refers to rich but noisy feedback that contains noise and cannot directly indicate the user's preferences.…”

Section: Introductionmentioning

confidence: 99%

Denoising User-aware Memory Network for Recommendation

Bian¹,

Zhou²,

Fu³

et al. 2021

Preprint

View full text Add to dashboard Cite

For better user satisfaction and business effectiveness, more and more attention has been paid to the sequence-based recommendation system, which is used to infer the evolution of users' dynamic preferences, and recent studies have noticed that the evolution of users' preferences can be better understood from the implicit and explicit feedback sequences. However, most of the existing recommendation techniques do not consider the noise contained in implicit feedback, which will lead to the biased representation of user interest and a suboptimal recommendation performance. Meanwhile, the existing methods utilize item sequence for capturing the evolution of user interest. The performance of these methods is limited by the length of the sequence, and can not effectively model the long-term interest in a long period of time. Based on this observation, we propose a novel CTR model named denoising user-aware memory network (DUMN). Specifically, the framework: (i) proposes a feature purification module based on orthogonal mapping, which use the representation of explicit feedback to purify the representation of implicit feedback, and effectively denoise the implicit feedback; (ii) designs a user memory network to model the long-term interests in a fine-grained way by improving the memory network, which is ignored by the existing methods; and (iii) develops a preference-aware interactive representation component to fuse the long-term and short-term interests of users based on gating to understand the evolution of unbiased preferences of users. Extensive experiments on two real e-commerce user behavior datasets show that DUMN has a significant improvement over the state-of-the-art baselines. The code of DUMN model has been uploaded as an additional material.

show abstract

“…However, except item-ids, category-ids and similar id features, both of them lack the modeling of finer grained features like color and materials. [5,8,12] exploit transformers or similar hierarchical attention structure to capture multiple interest representations of users. Yet, their complicated model structures make serving online nearly infeasible without specific optimization for practical usage.…”

Section: Introductionmentioning

confidence: 99%

Multiple Interest and Fine Granularity Network for User Modeling

Xie¹,

Wei²,

Hua³

et al. 2021

Preprint

View full text Add to dashboard Cite

User modeling plays a fundamental role in industrial recommender systems, either in the matching stage and the ranking stage, in terms of both the customer experience and business revenue. How to extract users' multiple interests effectively from their historical behavior sequences to improve the relevance and personalization of the recommend results remains an open problem for user modeling. Most existing deep-learning based approaches exploit item-ids and category-ids but neglect fine-grained features like color and material, which hinders modeling the fine granularity of users' interests.In the paper, we present Multiple interest and Fine granularity Network (MFN), which tackle users' multiple and fine-grained interests and construct the model from both the similarity relationship and the combination relationship among the users' multiple interests. Specifically, for modeling the similarity relationship, we leverage two sets of embeddings, where one is the fixed embedding from pretrained models (e.g. Glove) to give the attention weights and the other is trainable embedding to be trained with MFN together. For modeling the combination relationship, self-attentive layers are exploited to build the higher order combinations of different interest representations. In the construction of network, we design an interest-extract module using attention mechanism to capture multiple interest representations from user historical behavior sequences and leverage an auxiliary loss to boost the distinction of the interest representations. Then a hierarchical network is applied to model the attention relation between the multiple interest vectors of different granularities and the target item. We evaluate MFN on both public and industrial datasets. The experimental results demonstrate that the proposed MFN achieves superior performance than other existed representing methods.

show abstract

Deep Multifaceted Transformers for Multi-objective Ranking in Large-Scale E-commerce Recommender Systems

Cited by 70 publications

References 39 publications

PASTO: Strategic Parameter Optimization in Recommendation Systems -- Probabilistic is Better than Deterministic

PASTO: Strategic Parameter Optimization in Recommendation Systems -- Probabilistic is Better than Deterministic

Denoising User-aware Memory Network for Recommendation

Multiple Interest and Fine Granularity Network for User Modeling

Contact Info

Product

Resources

About