Multi-Task and Multi-Scene Unified Ranking Model for Online Advertising

Tan, Shulong; Li, Meifang; Zhao, Weijie; Zheng, Yandan; Pei, Xin; Li, Ping

doi:10.1109/bigdata52589.2021.9671920

Cited by 3 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-Domain Recommendation (Tan et al 2021;Xu et al 2023;Wang et al 2022;Zhang et al 2022b;Luo et al 2022;Gao et al 2023) aims to capture the commonalities and diversities of various scenarios with a unified model. In recent times, a multitude of relevant endeavors has emerged, propelling the advancement of this field.…”

Section: Related Work Multi-domain Recommendationmentioning

confidence: 99%

D3: A Methodological Exploration of Domain Division, Modeling, and Balance in Multi-Domain Recommendations

Jia,

Wang,

Lin

et al. 2024

AAAI

View full text Add to dashboard Cite

To enhance the efficacy of multi-scenario services in industrial recommendation systems, the emergence of multi-domain recommendation has become prominent, which entails simultaneous modeling of all domains through a unified model, effectively capturing commonalities and differences among them. However, current methods rely on manual domain partitioning, which overlook the intricate domain relationships and the heterogeneity of different domains during joint optimization, hindering the integration of domain commonalities and differences. To address these challenges, this paper proposes a universal and flexible framework D3 aimed at optimizing the multi-domain recommendation pipeline from three key aspects. Firstly, an attention-based domain adaptation module is introduced to automatically identify and incorporate domain-sensitive features during training. Secondly, we propose a fusion gate module that enables the seamless integration of commonalities and diversities among domains, allowing for implicit characterization of intricate domain relationships. Lastly, we tackle the issue of joint optimization by deriving loss weights from two complementary viewpoints: domain complexity and domain specificity, alleviating inconsistencies among different domains during the training phase. Experiments on three public datasets demonstrate the effectiveness and superiority of our proposed framework. In addition, D3 has been implemented on a real-life, high-traffic internet platform catering to millions of users daily.

show abstract

Section: Related Work Multi-domain Recommendationmentioning

confidence: 99%

D3: A Methodological Exploration of Domain Division, Modeling, and Balance in Multi-Domain Recommendations

Jia,

Wang,

Lin

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…For VLP model, we propose a novel pre-training framework called R2D2 for cross-modal learning. Inspired by industrial technology such as recommender systems [3,29] and online advertising [27], we apply global contrastive pre-ranking to obtain image-text representations and fine-grained ranking to further improve model performance. We also introduce a two-way distillation method, consisting of target-guided distillation and feature-guided distillation.…”

mentioning

confidence: 99%

Zero and R2D2: A Large-scale Chinese Cross-modal Benchmark and A Vision-Language Framework

Xie¹,

Cai²,

Song³

et al. 2022

Preprint

View full text Add to dashboard Cite

Vision-language pre-training (VLP) on large-scale datasets has shown premier performance on various downstream tasks. A complete and fair benchmark (i.e., including large-scale pre-training datasets and diverse downstream tasks) is essential for VLP. While there are plenty of benchmarks with English corpus, building a rich benchmark for VLP with other languages, such as Chinese, remains a critical problem. To this end, we build a large-scale Chinese cross-modal benchmark called Zero for the research community to fairly compare VLP models. We release two pre-training datasets and five fine-tuning datasets for downstream tasks. Alongside, we propose a novel pre-training framework of pre-Ranking + Ranking for cross-modal learning. Specifically, we apply global contrastive pre-ranking to learn the individual representations of images and texts, respectively. We then fuse the representations in a fine-grained ranking manner via an image-text cross encoder and a text-image cross encoder. To further enhance the capability of the model, we propose a two-way distillation strategy consisting of target-guided Distillation and feature-guided Distillation. For brevity, we name our model R2D2. We achieve state-of-the-art performance on four public cross-modal datasets and the proposed five downstream datasets. When conducting zero-shot tasks on Flickr30k-CN, COCO-CN, and MUGE, R2D2 pre-trained on a 250 million dataset achieves significant improvements of 4.7%, 5.4%, and 6.3% in mean recall compared to the state-of-the-art. The datasets, models, and codes are available at https://github.com/yuxie11/R2D2. IntroductionVision-language pre-training (VLP) mainly learns the semantic correspondence between vision and natural language. Seminal works [15,17,24,26,30] explore the VLP model and achieve significant improvement on various vision-language tasks, supported by massive data [25], excellent architectures such as Transformer [28], cross-modal models such as CLIP [24], hardware equipment, etc. In this paper, we focus on large-scale vision-language data and cross-modal learning.With a large-scale training corpus (mainly in English language), VLP models have shown to be beneficial for downstream tasks [15,21]. However, existing Chinese vision-language datasets are rare and with various limitations. For instance, M6-Corpus [20] is a multi-modal pre-training dataset but is still not publicly available. Wukong [9] is a newly published cross-modal pre-training dataset Preprint. Under review.

show abstract