The Impact of User Demographics and Task Types on Cross-App Mobile Search

Abstract

User interaction data is an important source of supervision in counterfactual learning to rank (CLTR). Such data suffers from presentation bias. Much work in unbiased learning to rank (ULTR) focuses on position bias, i.e., items at higher ranks are more likely to be examined and clicked. Inter-item dependencies also influence examination probabilities, with outlier items in a ranking as an important example. Outliers are defined as items that observably deviate from the rest and therefore stand out in the ranking. In this paper, we identify and introduce the bias brought about by outlier items: users tend to click more on outlier items and their close neighbors.To this end, we first conduct a controlled experiment to study the effect of outliers on user clicks. Next, to examine whether the findings from our controlled experiment generalize to naturalistic situations, we explore real-world click logs from an e-commerce platform. We show that, in both scenarios, users tend to click significantly more on outlier items than on non-outlier items in the same rankings. We show that this tendency holds for all positions, i.e., for any specific position, an item receives more interactions when presented as an outlier as opposed to a non-outlier item. We conclude from our analysis that the effect of outliers on clicks is a type of bias that should be addressed in ULTR. We therefore propose an outlieraware click model that accounts for both outlier and position bias, called outlier-aware position-based model (OPBM). We estimate click propensities based on OPBM; through extensive experiments performed on both real-world e-commerce data and semi-synthetic data, we verify the effectiveness of our outlier-aware click model. Our results show the superiority of OPBM against baselines in terms of ranking performance and true relevance estimation. CCS CONCEPTS• Information systems → Learning to rank.

Transitions in daily search tactics: during the cross-app interaction search process

“…Therefore, the app system should add user-selectable options for evaluating an individual item or list of items, such as including links to widespread queries on the portal's main page (Aliannejadi et al. , 2019, 2021), which can increase customer satisfaction. In this way, users can decide for themselves whether they want to browse the results list or view the details of the content presented by the current system.Prompt to organize search results optionally.…”

“…Users who need more information after starting a search are usually forced to evaluate search results or an individual item(Lead→EvalI or Lead→EvalR). Therefore, the app system should add user-selectable options for evaluating an individual item or list of items, such as including links to widespread queries on the portal's main page (Aliannejadi et al, 2019(Aliannejadi et al, , 2021, which can increase customer satisfaction. In this way, users can decide for themselves whether they want to browse the results list or view the details of the content presented by the current system.…”

“…They can also use search history aggregated from different APP systems to provide enhanced query recommendations (Church and Oliver, 2011). It has also been argued that need a unified APP search system that acts as a metasearch on the user's mobile device: it identifies the target app for the user's query, submits the query to the app and then presents the results to the user as expected (Aliannejadi et al, 2018a(Aliannejadi et al, , 2021Pragada et al, 2014).…”