Relevance ranking is not relevance ranking or, when the user is not the user, the search results are not search results

Bade, David

doi:10.1108/14684520710841793

Cited by 11 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evaluation of the search output refers to the assessment of the information objects encountered for relevance. The notion of evaluation is fraught with layers of complexity including the cognitive elements required for evaluation (Bade, 2007;Barry, 1998;Borlund, 2003;Fitzgerald & Galloway, 2001;Saracevic, 2007aSaracevic, , 2007bSchamber, 1991;Vakkari, 1999;Vakkari & Hakala, 2000). Users often spend more time determining the value of a given document through evaluation than the other types of information search activities (Xie, Benoit, & Zhang, 2010).…”

Section: Evaluation Of the Search Outputmentioning

confidence: 99%

Behavioral changes in transmuting multisession successive searches over the web

Lin

Xie

2013

J Am Soc Inf Sci Tec

View full text Add to dashboard Cite

Multisession successive information searches are common but little research has focused on quantitative analysis. This article enhances our understanding of successive information searches by employing an experimental method to observe whether and how the behavioral characteristics of searchers statistically significantly changed over sessions. It focuses on a specific type of successive search called transmuting successive searches, in which searchers learn about and gradually refine their information problems during the course of the information search. The results show that searchers' behavioral characteristics indeed exhibit different patterns in different sessions. The identification of the behavioral characteristics can help information retrieval systems to detect stages or sessions of the information search process. The findings also help validate a theoretical framework to explain successive searches and suggest system requirements for supporting the associated search behavior. The study is one of the first to not only test for statistical significance among research propositions concerning successive searches but to also apply the research principles of implicit relevance feedback to successive searches.

show abstract

Section: Evaluation Of the Search Outputmentioning

confidence: 99%

Behavioral changes in transmuting multisession successive searches over the web

Lin

Xie

2013

J Am Soc Inf Sci Tec

View full text Add to dashboard Cite

show abstract

“…This option was available in four of the eight databases used (ASSIA, ERIC, Scopus and SSCI). Relevance sorting refers to ‘various statistical methods for ordering documents matching a search term’ undertaken by the database software [39]. While exact formulae for relevance sorting are kept secret by the database provider companies, at a basic level they are algorithms designed to identify and match search terms with documents based on pre-defined criteria [40].…”

Section: Methodsmentioning

confidence: 99%

Systematically retrieving research in the digital age: Case study on the topic of social networking sites and young people’s mental health

Best

Taylor

Manktelow

et al. 2014

Journal of Information Science

View full text Add to dashboard Cite

Online information seeking has become normative practice among both academics and the general population. This study appraised the performance of eight databases to retrieve research pertaining to the influence of social networking sites on the mental health of young people. A total of 43 empirical studies on young people’s use of social networking sites and the mental health implications were retrieved. Scopus and SSCI had the highest sensitivity with PsycINFO having the highest precision. Effective searching requires large generic databases, supplemented by subject-specific catalogues. The methodology developed here may provide inexperienced searchers, such as undergraduate students, with a framework to define a realistic scale of searching to undertake for a particular literature review or similar project.

show abstract

“…Borlund (2003) outlines a "relevance framework," based on the many different conceptions of relevance in previous research including, "classes, types, degrees, criteria, and levels of relevance" (p. 923). Furthermore, Bade (2007) argued against misinterpretations of relevance evaluation arguing for a multidimensional understanding over a binary one. Viewing relevance as either objective or subjective leads to two major flaws: "The first is that any relevance judgment by any human being will always be subjective to some degree, no matter Search result list evaluation how objective that person may strive to be.…”

Section: Criteria -Lists and Documentsmentioning

confidence: 99%

“…Previous studies focused more on relevance criteria than elements or other dimensions of evaluation. Studies comparing both list and document evaluation criteria focus on relevance (Bade, 2007;Borlund, 2003;Saracevic, 2007a,b;Savolainen and Kari, 2006;Vakkari and Hakala, 2000). Search result list evaluation receives significantly less attention than document evaluation.…”

Section: Introductionmentioning

confidence: 99%

Search result list evaluation versus document evaluation: similarities and differences

Xie

Benoit²

2013

Journal of Documentation

View full text Add to dashboard Cite

Purpose-The purpose of this study is to compare the evaluation of search result lists and documents, in particular evaluation criteria, elements, association between criteria and elements, pre/post and evaluation activities, and the time spent on evaluation. Design/methodology/approach-The study analyzed the data collected from 31 general users through prequestionnaires, think aloud protocols and logs, and post questionnaires. Types of evaluation criteria, elements, associations between criteria and elements, evaluation activities and their associated pre/post activities, and time were analyzed based on open coding. Findings-The study identifies the similarities and differences of list and document evaluation by analyzing 21 evaluation criteria applied, 13 evaluation elements examined, pre/post and evaluation activities performed and time spent. In addition, the authors also explored the time spent in evaluating lists and documents for different types of tasks. Research limitations/implications-This study helps researchers understand the nature of list and document evaluation. Additionally, this study connects elements that participants examined to criteria they applied, and further reveals problems associated with the lack of integration between list and document evaluation. The findings of this study suggest more elements, especially at list level, be available to support users applying their evaluation criteria. Integration of list and document evaluation and integration of pre, evaluation and post evaluation activities for the interface design is the absolute solution for effective evaluation. Originality/value-This study fills a gap in current research in relation to the comparison of list and document evaluation.

show abstract

Relevance ranking is not relevance ranking or, when the user is not the user, the search results are not search results

Cited by 11 publications

References 6 publications

Behavioral changes in transmuting multisession successive searches over the web

Behavioral changes in transmuting multisession successive searches over the web

Systematically retrieving research in the digital age: Case study on the topic of social networking sites and young people’s mental health

Search result list evaluation versus document evaluation: similarities and differences

Contact Info

Product

Resources

About