Rank by Time or by Relevance?

Carmel, David; Halawi, Guy; Lewin-Eytan, Liane; Maarek, Yoelle; Raviv, Ariel

doi:10.1145/2806416.2806471

Cited by 38 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In real-world applications like search engines and recommendation systems, systems provide ranked lists tailored to users and their queries [1,5,20]. In some cases, mapping those preferences into an ordinal variable leads to better user experience.…”

Section: Related Work and Discussionmentioning

confidence: 99%

L2P: An Algorithm for Estimating Heavy-tailed Outcomes

Wang,

Varol,

Eliassi-Rad

2019

Preprint

View full text Add to dashboard Cite

Many real-world prediction tasks have outcome (a.k.a. target or response) variables that have characteristic heavy-tail distributions. Examples include copies of books sold, auction prices of art pieces, etc. By learning heavy-tailed distributions, "big and rare" instances (e.g., the best-sellers) will have accurate predictions. Most existing approaches are not dedicated to learning heavy-tailed distribution; thus, they heavily under-predict such instances. To tackle this problem, we introduce Learning to Place (L2P), which exploits the pairwise relationships between instances to learn from a proportionally higher number of rare instances. L2P consists of two stages. In Stage 1, L2P learns a pairwise preference classifier: is instance A > instance B?. In Stage 2, L2P learns to place a new instance into an ordinal ranking of known instances. Based on its placement, the new instance is then assigned a value for its outcome variable. Experiments on real data show that L2P outperforms competing approaches in terms of accuracy and capability to reproduce heavy-tailed outcome distribution. In addition, L2P can provide an interpretable model with explainable outcomes by placing each predicted instance in context with its comparable neighbors.

show abstract

Section: Related Work and Discussionmentioning

confidence: 99%

L2P: An Algorithm for Estimating Heavy-tailed Outcomes

Wang,

Varol,

Eliassi-Rad

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, more studies have been conducted with real queries and judgments derived from users' interactions with the results in the context of large Web email services. Some work has studied the effect of displaying results ranked based on relevance or recency [9,10,25]. Carmel et al [9] followed a two-phase ranking scheme where in the first stage recall is emphasized based on exact or partial term matching, and in the second stage, a learning-to-rank (LTR) model is used to balance the relevance and recency for ranking.…”

Section: Related Workmentioning

confidence: 99%

“…Some work has studied the effect of displaying results ranked based on relevance or recency [9,10,25]. Carmel et al [9] followed a two-phase ranking scheme where in the first stage recall is emphasized based on exact or partial term matching, and in the second stage, a learning-to-rank (LTR) model is used to balance the relevance and recency for ranking. This way outperforms the widely adopted time-based ranking by a large margin.…”

Section: Related Workmentioning

confidence: 99%

“…We train the LambdaMart models using LightGBM 9 . We set the number of trees to be 500, the number of leaves in each tree as 150, the shrinkage rate as 0.3, the objective as LambdaRank, and the rounds of early-stop based on the validation performance as 30.…”

Section: Training Settingsmentioning

confidence: 99%

“…Email has long been an indispensable way for both business and personal communication in our daily life. As email storage quotas become large due to cheap data storage, users do not feel the need to delete emails and let their mailboxes keep growing [9,20]. Therefore, an effective email search engine is quite important to support users' search needs and facilitate users' daily life.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Leveraging User Behavior History for Personalized Email Search

Metrikov

et al. 2021

Proceedings of the Web Conference 2021

View full text Add to dashboard Cite

An effective email search engine can facilitate users' search tasks and improve their communication efficiency. Users could have varied preferences on various ranking signals of an email, such as relevance and recency based on their tasks at hand and even their jobs. Thus a uniform matching pattern is not optimal for all users. Instead, an effective email ranker should conduct personalized ranking by taking users' characteristics into account. Existing studies have explored user characteristics from various angles to make email search results personalized. However, little attention has been given to users' search history for characterizing users. Although users' historical behaviors have been shown to be beneficial as context in Web search, their effect in email search has not been studied and remains unknown. Given these observations, we propose to leverage user search history as query context to characterize users and build a context-aware ranking model for email search. In contrast to previous context-dependent ranking techniques that are based on raw texts, we use ranking features in the search history. This frees us from potential privacy leakage while giving a better generalization power to unseen users. Accordingly, we propose a context-dependent neural ranking model (CNRM) that encodes the ranking features in users' search history as query context and show that it can significantly outperform the baseline neural model without using the context. We also investigate the benefit of the query context vectors obtained from CNRM on the state-of-the-art learning-to-rank model LambdaMart by clustering the vectors and incorporating the cluster information. Experimental results show that significantly better results can be achieved on LambdaMart as well, indicating that the query clusters can characterize different users and effectively turn the ranking model personalized. CCS CONCEPTS• Information systems → Environment-specific retrieval; Retrieval models and ranking.

show abstract

On-Device Query Auto-completion for Email Search

Qiao,

Godwin,

Ouyang

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Traditional query auto-completion (QAC) relies heavily on search logs collected over many users. However, in on-device email search, the scarcity of logs and the governing privacy constraints make QAC a challenging task. In this work, we propose an on-device QAC method that runs directly on users’ devices, where users’ sensitive data and interaction logs are not collected, shared, or aggregated through web services. This method retrieves candidates using pseudo relevance feedback, and ranks them based on relevance signals that explore the textual and structural information from users’ emails. We also propose a private corpora based evaluation method, and empirically demonstrate the effectiveness of our proposed method.

show abstract

Rank by Time or by Relevance?

Cited by 38 publications

References 16 publications

L2P: An Algorithm for Estimating Heavy-tailed Outcomes

L2P: An Algorithm for Estimating Heavy-tailed Outcomes

Leveraging User Behavior History for Personalized Email Search

On-Device Query Auto-completion for Email Search

Contact Info

Product

Resources

About