Risk-Sensitive Learning to Rank with Evolutionary Multi-Objective Feature Selection

Abstract

Learning to Rank (L2R) is one of the main research lines in Information Retrieval. Risk-sensitive L2R is a sub-area of L2R that tries to learn models that are good on average while at the same time reducing the risk of performing poorly in a few but important queries (e.g., medical or legal queries). One way of reducing risk in learned models is by selecting and removing noisy, redundant features, or features that promote some queries to the detriment of others. This is exacerbated by learning methods that usually maximize an average metric (e.g., mean average precision (MAP) or Normalized Discounted Cumulative Gain (NDCG)). However, historically, feature selection (FS) methods have focused only on effectiveness and feature reduction as the main objectives. Accordingly, in this work, we propose to evaluate FS for L2R with an additional objective in mind, namely risk-sensitiveness . We present novel single and multi-objective criteria to optimize feature reduction, effectiveness, and risk-sensitiveness, all at the same time. We also introduce a new methodology to explore the search space, suggesting effective and efficient extensions of a well-known Evolutionary Algorithm (SPEA2) for FS applied to L2R. Our experiments show that explicitly including risk as an objective criterion is crucial to achieving a more effective and risk-sensitive performance. We also provide a thorough analysis of our methodology and experimental results.

Effective Lightweight Learning-to-Rank Method Using Unified Term Impacts

“…We also plan to study the effect of methods developed to produce more stable rankings, namely, risk-sensitive L2R methods, and study their impact when applied to UTI methods. Risk-sensitivity is a subarea of L2R that tries to learn models that are good on average while at the same time reducing the risk of performing poorly in a few but important queries (e.g., medical or legal queries) [5]. Their usage when combined with UTI values represents a challenge, since when computing UTI values, the learning is performed at indexing times.…”

“…Fig. 1 shows query processing performed through a two-step L2R-based search engine [2]- [5]. In the first step, top-k ranking results are retrieved using a low-cost ranking strategy, such as…”

“…While other architectures could be adopted, such as one that involves processing all documents with the L2R method, the two-step approach described in Fig. 1 is mentioned and adopted in the literature as a solution to allow for fast query processing since the top ranker needs to process the features of only a few documents [2]- [5]. The approach is also useful to create L2R reference collections.…”

Jusbrasil e os Desafios Tecnológicos para Facilitar e Aprimorar o Acesso à Justiça

“…Fornecer resultados precisosé uma tarefa de vital importância e por isso perseguimos continuamente novos algoritmos e métodos para processamento de consultas eficazes e eficientes. Entre as necessidades voltadas a eficácia na entrega de resultados relevantes, são importantes estudos voltados ao uso de aprendizagem de máquina em sistemas de busca [Silva et al 2009, Sousa et al 2019. A empresa tem investido nessa frente, bem como em infraestrutura para a coleta e disponibilização de informação que possa ser usada como insumo para que os algoritmos de aprendizagem de máquina possam gerar ordenação de resposta de alta qualidade, tais como dados de navegação, agrupamentos de usuários e informação demográfica, dentre outros.…”

Evolutionary Risk-Sensitive Feature Selection for Learning to Rank

“…During the dissertation, we published some papers in the world leading In-formation Retrieval conferences and journals, such as [Sousa et al 2016](A1) and [Sousa et al 2019](A2). Beside them, we also published other papers in L2R area, as [Sousa et al 2012](B1), [Freitas et al 2016](B3) 3 , and [Freitas et al 2018](A2).…”