How Does Ranking Affect User Choice in Online Search?

Abstract

This paper investigates whether a search engine's ordering of algorithmic results has an important effect on website traffic. A website's ranking on a search engine results page is positively correlated with the clicks that it receives. This could result from the search engine's accurately predicting the websites relevance to users. Or it could result from users merely clicking on the highest ranked links, regardless of the website's relevance. Using a unique dataset, we find that a website's rank, not just The analysis for this paper was performed in conjunction with ongoing work for Microsoft Corporation; Keystone and Seabright acknowledge financial support from Microsoft. The primary data that were used in the analysis come from log files from Microsoft's Bing search engine and include the website names, search result rankings, and click-through rates for results that were presented in response to user search queries on Bing.com. Supplementary data for additional website click-through rates were derived from additional Microsoft opt-in consumer panel data that contained online behavior. The authors are grateful to Elan Fuld, who provided excellent research assistance; to Scott Gingold, who was involved with the project from the beginning and gave us invaluable feedback, ideas, encouragement and support; and to Thierry Magnac, who gave us invaluable econometric expertise and spent much time helping us to understand the data.M. Glick Department of Economics, University of Utah, Salt Lake City, UT 84112, USA e-mail: glick@economics.utah.edu its relevance, strongly and significantly affects the likelihood of a click. We also find evidence that rank influences CTRs partly by controlling access to the scarce attention of users, but primarily by substituting the reputational capital of the search engine for the reputation of individual websites.

Opinion Dynamics via Search Engines (and Other Algorithmic Gatekeepers)

“…Moreover, we assume they can process the ranked list, subject to limitations of attention, meaning they implicitly favor higher ranked websites due to attention bias. Indeed, as shown by Pan et al (2007); Glick et al (2011);Yom-Tov et al (2013); Epstein and Robertson (2015), keeping all other things equal (e.g., the fit of a given website with respect to the individual's preferences), highest ranked websites tend to receive significantly more "attention" by users than lower ranked ones. 9 Since we model the ranking (r n ) as a probability 7 See Yom-Tov et al (2013); Flaxman et al (2016); White and Horvitz (2015) for evidence of individual preference for like-minded news in the context of search engines.…”

“…Finally, the paper is also related to the literature outside economics discussing the possible implications of the search engines' architecture (or, more generally, of algorithmic gatekeeping) on democratic outcomes. This literature encompasses communication scholars (Hargittai, 2004;Granka, 2010), legal scholars (Goldman, 2006;Grimmelmann, 2009;Sunstein, 2009), media activists (Pariser, 2011), psychologists (Epstein and Robertson, 2015), political scientists (Putnam, 2001;Hindman, 2009;Lazer, 2015), sociologists (Tufekci, 2015) and, last but not least, computer scientists (Cho et al, 2005;Menczer et al, 2006;Pan et al, 2007;Glick et al, 2011;Flaxman et al, 2016;Bakshy et al, 2015).…”

The impact of positioning on click-through-rates in hotel metasearch engines

“…In the online environment, there exists ample evidence of position effects. In the context of search engine results pages (SERPs), a higher-positioned website is associated with better visibility (Drèze and Zufryden, 2004) and a higher likelihood of a click (Glick et al, 2014; Murphy et al, 2006). The positioning also matters in advertising listings on SERPs, which is considered the most investigated amongst the literature of search results (Schultz, 2020).…”

“…Besides, although there is previous literature analysing the effects of positioning in different online contexts (e.g. Glick et al, 2014 or Park and Agarwal 2018), hotel metasearch engines organise, structure and visualise results in a different manner than previously studied general search engines or leading eCommerce websites. This fact might also influence the impact of positioning on the customer and could therefore result in different, unconscious or spontaneous decisions when clicking on one provider or another.…”

Cheating in Ranking Systems

“…It has been shown that a website's rank, not just its relevance, strongly and significantly affects the likelihood of a click (Glick et al, 2014).…”