One Model to Serve All

Sheng, Xiang-Rong; Zhao, Lei; Zhou, Guorui; Ding, Xinyao; Dai, Binding; Luo, Qiang; Yang, Shiyou; Lv, Jingshan; Zhang, Chi; Deng, Hongbo; Zhu, Xiaoqiang

doi:10.1145/3459637.3481941

Cited by 122 publications

(27 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the challenges encountered by multi-task learning are more complicated in the multi-domain scenario. The task label space of different input data sources is not consistent, so all domains sharing one multi-task model will cause the "multi-domain seesaw" problem [31]. In real industrial scenarios, the cost of training different multi-task models in different domains is unacceptable, and it is impossible to explore correlations between different domains to improve performance.…”

Section: Relate Workmentioning

confidence: 99%

“…However, mixing all the data directly and training with a unified model ignores the differences between domains and tasks. The inability to align embeddings with different semantics will result in domain seesaw [31] due to the different distributions of user behaviors and item candidates in multiple scenarios. Since different targets have distinctive sparsity and influence each other, the inability to balance the targets of multiple tasks can lead to the task seesaw [33].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information

Chang¹,

Zhang²,

Hui³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the increase of content pages and display styles in online services such as online-shopping and video-watching websites, industrial-scale recommender systems face challenges in multidomain and multi-task recommendations. The core of multi-task and multi-domain recommendation is to accurately capture user interests in different domains given different user behaviors. In this paper, we propose a plug-and-play Parameter and Embedding Personalized Network (PEPNet) for multi-task recommendation in the multi-domain setting. PEPNet takes features with strong biases as input and dynamically scales the bottom-layer embeddings and the top-layer DNN hidden units in the model through a gate mechanism. By mapping personalized priors to scaling weights ranging from 0 to 2, PEPNet introduces both parameter personalization and embedding personalization. Embedding Personalized Network (EPNet) selects and aligns embeddings with different semantics under multiple domains. Parameter Personalized Network (PPNet) influences DNN parameters to balance interdependent targets in multiple tasks. We have made a series of special engineering optimizations combining the Kuaishou training framework and the online deployment environment. We have deployed the model in Kuaishou apps, serving over 300 million daily users. Both online and offline experiments have demonstrated substantial improvements in multiple metrics. In particular, we have seen a more than 1% online increase in three major scenarios.

show abstract

Section: Relate Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information

Chang¹,

Zhang²,

Hui³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…More specifically, each scenario has a domain layer that only uses its own data to adjust parameters. Another classical approach STAR [20] added an extra shared tower and proposed a novel parameter fusion method. It multiplies the shared parameters of the extra tower and the customized parameter matrix of the specific scenario to obtain the final network processing parameters.…”

Section: Multi-tower Paradigmmentioning

confidence: 99%

“…• STAR [20] proposes a star topology to accommodate with the scenario-specific characteristics. Specifically, a shared network works as the center node for knowledge sharing and each scenario network connects only with the center node.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…That is, above the shared bottom layer for feature transformation, different network structures on top of the former are searched. While some of them calls for domain specific expertise [15,19,20], the others choose to learn the structure automatically [9,21,29]. Though fruitful performance gain is obtained by these solutions, they all take it for granted that the representations generated by the bottom layer are fair for different scenarios.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Scenario Ranking with Adaptive Feature Learning

Tian,

Li,

Chen

et al. 2023

Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

Recently, Multi-Scenario Learning (MSL) is widely used in recommendation and retrieval systems in the industry because it facilitates transfer learning from different scenarios, mitigating data sparsity and reducing maintenance cost. These efforts produce different MSL paradigms by searching more optimal network structure, such as Auxiliary Network, Expert Network, and Multi-Tower Network. It is intuitive that different scenarios could hold their specific characteristics, activating the user's intents quite differently. In other words, different kinds of auxiliary features would bear varying importance under different scenarios. With more discriminative feature representations refined in a scenario-aware manner, better ranking performance could be easily obtained without expensive search for the optimal network structure. Unfortunately, this simple idea is mainly overlooked but much desired in real-world systems.To this end, in this paper, we propose a multi-scenario ranking framework with adaptive feature learning (named Maria). Specifically, Maria is devised to inject the scenario semantics in the bottom part of the network to derive more discriminative feature representations. There are three components designed in Maria * Corresponding author. Work done when Yu Tian was an intern at Alibaba.

show abstract