Multi-Aspect Dense Retrieval

Kong, Weize; Khadanga, Swaraj; Li, Cheng; Gupta, Shaleen; Zhang, Mingyang; Xu, Wensong; Bendersky, Michael

doi:10.1145/3534678.3539137

Cited by 17 publications

(21 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The success of e-commerce platforms depends on matching the query to the appropriate items. Users visit these platforms and enter search queries to retrieve their desired items [11,20]. Therefore, matching the query to the relevant items is essential for the success of e-commerce platforms [7].…”

Section: Related Workmentioning

confidence: 99%

“…To better learn the relative position of sentence vectors in the semantic space, [2,8,37] provided difficult negative samples for contrastive learning through optimizing the sampling strategy. These works generally focused only on the pure input text of query and item, neglecting the additional attribute information along with the text, which is proved to be useful in e-commerce tasks [11,33]. [40] extracted keywords and abstract intents from sentences and performed semantic matching at different granularities.…”

Section: Related Workmentioning

confidence: 99%

“…To name a few, [40] extracted keywords and intents from the input texts as attributes, and enhances the matching from different granularities. [11] treated multiple additional inputs as attributes, and explicitly represented each attribute as an embedding, which is further fused into the input embeddings. Most recently, [31] considered the relationship between attributes and input text by introducing an attribute fusion layer to identify the most relevant features.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Intent Attribute-Aware Text Matching in Searching

Li,

Chen,

Xiang

et al. 2024

Proceedings of the 17th ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Text matching systems have become a fundamental service in most Searching platforms. For instance, they are responsible for matching user queries to relevant candidate items, or rewriting the user-input query to a pre-selected high-performing one for a better search experience. In practice, both the queries and items often contain multiple attributes, such as the category of the item and the location mentioned in the query, which represent condensed key information that is helpful for matching. However, most of the existing works downplay the effectiveness of attributes by integrating them into text representations as supplementary information. Hence, in this work, we focus on exploring the relationship between the attributes from two sides. Since attributes from two ends are often not aligned in terms of number and type, we propose to exploit the benefit of attributes by multiple-intent modeling. The intents extracted from attributes summarize the diverse needs of queries and provide rich content of items, which are more refined and abstract, and can be aligned for paired inputs. Concretely, we propose a multi-intent attribute-aware matching model (MIM), which consists of three main components: attribute-aware encoder, multiintent modeling, and intent-aware matching. In the attribute-aware encoder, the text and attributes are weighted and processed through a scaled attention mechanism with regard to the attributes' importance. Afterward, the multi-intent modeling extracts intents from two ends and aligns them. Herein, we come up with a distribution loss to ensure the learned intents are diverse but concentrated, and a kullback-leibler divergence loss that aligns the learned intents. Finally, in the intent-aware matching, the intents are evaluated by a self-supervised masking task, and then incorporated to output the final matching result. Extensive experiments on three real-world

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Intent Attribute-Aware Text Matching in Searching

Li,

Chen,

Xiang

et al. 2024

Proceedings of the 17th ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

show abstract

“…After low level networks 𝐿 0 (embedding tables) and 𝐿 1 (feature encoding layers) in Figure 1, we fuse different aspects of query and item as in literature [12]. Each aspect 𝐸 𝑎 represents some fine-grained properties of query and item, such as ID, text and sparse features.…”

Section: Aspect Gating Fusionmentioning

confidence: 99%

An Unified Search and Recommendation Foundation Model for Cold-Start Scenario

Gong,

Ding,

et al. 2023

Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

In modern commercial search engines and recommendation systems, data from multiple domains is available to jointly train the multi-domain model. Traditional methods train multi-domain models in the multi-task setting, with shared parameters to learn the similarity of multiple tasks, and task-specific parameters to learn the divergence of features, labels, and sample distributions of individual tasks. With the development of large language models, LLM can extract global domain-invariant text features that serve both search and recommendation tasks. We propose a novel framework called S&R Multi-Domain Foundation, which uses LLM to extract domain invariant features, and Aspect Gating Fusion to merge the ID feature, domain invariant text features and task-specific heterogeneous sparse features to obtain the representations of query and item. Additionally, samples from multiple search and recommendation scenarios are trained jointly with Domain Adaptive Multi-Task module to obtain the multi-domain foundation model. We apply the S&R Multi-Domain foundation model to cold start scenarios in the pretrain-finetune manner, which achieves better performance than other SOTA transfer learning methods. The S&R Multi-Domain Foundation model has been successfully deployed in Alipay Mobile Application's online services, such as content query recommendation and service card recommendation, etc.

show abstract

“…In this work, we focus on query and document representations in the retrieval setting. Some prior work, as a way to achieve semantically richer representations, model queries and documents using a combination of multiple vectors [26,43,61]. While such representations were shown to lead to better retrieval effectiveness, they do come at significant computational and storage costs.…”

Section: Related Workmentioning

confidence: 99%

Retrieval-Enhanced Machine Learning

Zamani

Dı́az

Dehghani³

et al. 2022

Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Dense retrieval models use bi-encoder network architectures for learning query and document representations. These representations are often in the form of a vector representation and their similarities are often computed using the dot product function. In this paper, we propose a new representation learning framework for dense retrieval. Instead of learning a vector for each query and document, our framework learns a multivariate distribution and uses negative multivariate KL divergence to compute the similarity between distributions. For simplicity and efficiency reasons, we assume that the distributions are multivariate normals and then train large language models to produce mean and variance vectors for these distributions. We provide a theoretical foundation for the proposed framework and show that it can be seamlessly integrated into the existing approximate nearest neighbor algorithms to perform retrieval efficiently. We conduct an extensive suite of experiments on a wide range of datasets, and demonstrate significant improvements compared to competitive dense retrieval models.

show abstract

Multi-Aspect Dense Retrieval

Cited by 17 publications

References 15 publications

Multi-Intent Attribute-Aware Text Matching in Searching

Multi-Intent Attribute-Aware Text Matching in Searching

An Unified Search and Recommendation Foundation Model for Cold-Start Scenario

Retrieval-Enhanced Machine Learning

Contact Info

Product

Resources

About