Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments

Li, Jiang; Sanderson, Mark; Willett, Peter; Norris, Michael; Oppenheim, Charles

doi:10.1016/j.joi.2010.06.005

Cited by 62 publications

(50 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Harzing, [17], Li et al, [12]) that Google Scholar is a very comprehensive bibliometric tool for computer science. It further develops this work by detailed comparisons of both paper counts and citations counts between Scopus General and Scopus More, and their combined results against Google Scholar.…”

Section: Which Bibliometric Database Is the Best?mentioning

confidence: 99%

A bibliometric study of Video Retrieval Evaluation Benchmarking (TRECVid): A methodological analysis

Thornley

McLoughlin

Johnson

et al. 2011

Journal of Information Science

View full text Add to dashboard Cite

This paper provides a discussion and analysis of methodological issues encountered during a scholarly impact and bibliometric study within the field of computer science (TRECVid Text Retrieval and Evaluation Conference, Video Retrieval Evaluation). The purpose of this paper is to provide a reflection and analysis of the methods used to provide useful information and guidance for those who may wish to undertake similar studies, and is of particular relevance for the academic disciplines which have publication and citation norms that may not perform well using traditional tools. Scopus and Google Scholar are discussed and a detailed comparison of the effects of different search methods and cleaning methods within and between these tools for subject and author analysis is provided. The additional database capabilities and usefulness of "Scopus More" in addition to "Scopus General" is discussed and evaluated. Scopus paper coverage is found to favourably compare to Google Scholar but Scholar consistently has superior performance at finding citations to those papers. These additional citations significantly increase the citation totals and also change the relative ranking of papers. Publish or Perish (PoP), a software wrapper for Google Scholar, is also examined and its limitations and some possible solutions are described. Data cleaning methods, including duplicate checks, expert domain checking of bibliographic data, and content checking of retrieved papers are compared and their relative effects on paper and citation count discussed. Google Scholar and Scopus are also compared as tools for collecting bibliographic data for visualisations of developing trends and, due to the comparative ease of collecting abstracts, Scopus is found far more effective.

show abstract

Section: Which Bibliometric Database Is the Best?mentioning

confidence: 99%

A bibliometric study of Video Retrieval Evaluation Benchmarking (TRECVid): A methodological analysis

Thornley

McLoughlin

Johnson

et al. 2011

Journal of Information Science

View full text Add to dashboard Cite

show abstract

“…High citations for some of these items caused the difference. (Li et al 2010). We all publish predominantly in journals, so no major improvements in h scores from PoP followed by CleanPoP were apparent in our data, with the exception of Author 6 who benefitted from PoP followed by CleanPoP's coverage of books, despite these constituting a minority of his publications.…”

mentioning

confidence: 59%

Users beware: implications of database errors when assessing the individual research records of ecologists and conservation biologists

Calver¹,

Beatty²,

Bryant³

et al. 2013

Pac. Conserv. Biol.

View full text Add to dashboard Cite

show abstract

“…Sanderson (2008), who calculated the h-index in detail for 3 British researchers, concluded that, after correcting the errors, the h-index had been underestimated by 5-10%. Li et al (2010), who also acknowledged the excessive data processing time required by Google Scholar, showed that data cleaning processes have, after all, little effect on results, something that had already been partially demonstrated by Baneyx (2008), albeit with very small samples. Doğan et al (2016) were the first to systematically calculate various indicators before and after cleaning the data (in this case in Google Scholar Metrics).…”

Section: Discussionmentioning

confidence: 97%

“…Other works of great interest, such as those by Harzing and Van der Wal (2008), Baneyx (2008), Li et al (2010), Adriaanse and Rensleigh (2011;2013), and De This means that, in general terms, scholarly literature about errors in Google Scholar, particularly articles focusing on the use of this tool in bibliometric analysis, is scarce, excessively fragmented and diffuse. There are no studies in which research designs have been specifically developed not only to identify but also to quantify the errors and evaluate their consequences.…”

Section: Discussionmentioning

confidence: 99%

Google Scholar como una fuente de evaluación científica: una revisión bibliográfica sobre errores de la base de datos

Orduña-Malea

Martín-Martín

López-Cózar

2017

Rev. esp. doc. cient.

View full text Add to dashboard Cite

Google Scholar (GS) is an academic search engine and discovery tool launched by Google (now Alphabet) in November 2004. The fact that GS provides the number of citations received by each article from all other indexed articles (regardless of their source) has led to its use in bibliometric analysis and academic assessment tasks, especially in social sciences and humanities. However, the existence of errors, sometimes of great magnitude, has provoked criticism from the academic community. The aim of this article is to carry out an exhaustive bibliographical review of all studies that provide either specific or incidental empirical evidence of the errors found in Google Scholar. The results indicate that the bibliographic corpus dedicated to errors in Google Scholar is still very limited (n= 49), excessively fragmented, and diffuse; the findings have not been based on any systematic methodology or on units that are comparable to each other, so they cannot be quantified, or their impact analysed, with any precision. Certain limitations of the search engine itself (time required for data cleaning, limit on citations per search result and hits per query) may be the cause of this absence of empirical studies.Keywords: Google Scholar; Academic search engines; Bibliographic databases; Errors; Quality.Google Scholar como una fuente de evaluación científica: una revisión bibliográfica sobre errores de la base de datos Resumen: Google Scholar es un motor de búsqueda académico y herramienta de descubrimiento lanzada por Google (ahora Alphabet) en noviembre de 2004. El hecho de que para cada registro bibliográfico se proporcione información acerca del número de citas recibidas por dicho registro desde el resto de registros indizados en el sistema (independientemente de su fuente) ha propiciado su utilización en análisis bibliométricos y en procesos de evaluación de la actividad académica, especialmente en Ciencias Sociales y Humanidades. No obstante, la existencia de errores, en ocasiones de gran magnitud, ha provocado su rechazo y crítica por una parte de la comunidad científica. Este trabajo pretende precisamente realizar una revisión bibliográfica exhaustiva de todos los estudios que de forma monográfica o colateral proporcionan alguna evidencia empírica sobre cuáles son los errores cometidos por Google Scholar (y productos derivados, como Google Scholar Metrics y Google Scholar Citations). Los resultados indican que el corpus bibliográfico dedicado a los errores en Google Scholar es todavía escaso (n= 49), excesivamente fragmentado, disperso, con resultados obtenidos sin metodologías sistemáticas y en unidades no comparables entre sí, por lo que su cuantificación y su efecto real no pueden ser caracterizados con precisión. Ciertas limitaciones del propio buscador (tiempo requerido de limpieza de datos, límite de citas por registro y resultados por consulta) podrían ser la causa de esta ausencia de trabajos empíricos. octubre-diciembre 2017octubre-diciembre , e185 ISSN-L:0210-0614. doi: http://dx.doi.org/10....

show abstract

Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments

Cited by 62 publications

References 29 publications

A bibliometric study of Video Retrieval Evaluation Benchmarking (TRECVid): A methodological analysis

A bibliometric study of Video Retrieval Evaluation Benchmarking (TRECVid): A methodological analysis

Users beware: implications of database errors when assessing the individual research records of ecologists and conservation biologists

Google Scholar como una fuente de evaluación científica: una revisión bibliográfica sobre errores de la base de datos

Contact Info

Product

Resources

About