Global Distribution of Google Scholar Citations: A Size-independent Institution-based Analysis

Abstract

Most currently available schemes for performance-based ranking of Universities/ Research organizations, such as, Quacarelli Symonds (QS), Times Higher Education (THE), Shanghai University-based All Research of World Universities (ARWU) use a variety of criteria that include productivity, citations, awards, reputation, etc., while Leiden and Scimago use only bibliometric indicators. The research performance evaluation in the aforesaid cases is based on bibliometric data from Web of Science or Scopus, which are commercially available priced databases. The coverage includes peer-reviewed journals and conference proceedings. Google Scholar (GS) on the other hand, provides a free and open alternative to obtaining citations of papers available on the net, (though it is not clear exactly which journals are covered.) Citations are collected automatically from the net and also added to self-created individual author profiles under Google Scholar Citations (GSC). This data was used by Webometrics Lab, Spain to create a ranked list of 4000+ institutions in 2016, based on citations from only the top 10 individual GSC profiles in each organization. (GSC excludes the top paper for reason,s explained in the text; the simple selection procedure makes the ranked list size-independent as claimed by the Cybermetrics Lab). Using this data (Transparent Ranking TR, 2016), we find the regional and country-wise distribution of GS-TR Citations. The size-independent ranked list is subdivided into deciles of 400 institutions each and the number of institutions and citations of each country obtained for each decile. We test for correlation between institutional ranks between GS-TR and the other ranking schemes for the top 20 institutions. Finally, we discuss our results in the context of questions like (1) Is it necessary to have one more global ranking scheme? (2) What are the likely benefits of the GS size-independent formulation? (3) What are the likely sources of error? and (4) Whether a truncated sample as in GS can indeed give a representative ranking acceptable at the global level?

A Comparative Analysis between Global University Rankings and Environmental Sustainability of Universities

“…This might be due to science being abstract and is much more difficult to measure tan any physical product, which has led to studies that have compared the positions that universities occupy in rankings. Some such studies have found a high correlation among the positions occupied by universities in ARWU, QS, and THE [2,3,7,8], and a low correlation of three GURs with Webometrics [3]. However, this work analyzed the geographical distribution of the universities in the Top 500 of the four studied GURs and evidences a different occupancy in them according to regions either for each quartile or the total of universities.…”

“…Finally, the Webometrics ranking has been analyzed with the three other GURs by two works. One work [3] has studied the correlation among the four GURs, and that with two others, namely Leiden and Scimago. As expected, the higher correlations appear among ARWU, THE and QS, and among Webometrics, Leiden and Scimago.…”

“…One reason is that it is impossible to measure "a university's quality" by employing only some relevant data, methodologies, and indicators [47]. As we have previously seen, GURs use indicators [4][5][6][7]36,38,48,49] that center mainly on research performance [3,50] and tend to rely on internationally accessible bibliometric databases and reputation surveys [50]. It is necessary to bear other different measures in mind, such as qualitative evaluations (pairwise) that must be done by experts in a given field [47].…”

“…Partiality toward research-related universities and lack of indicators related to teaching are other criticisms made of them [3,49]. Teaching and research are traditionally perceived as two functions that go hand-in-hand and are mutually supported [7].…”

Types of Errors Hiding in Google Scholar Data

“…Indeed, WoS has an average error rate of 0.1% [ 4 , 18 , 31 , 32 ], and this rate is 1.0% for Scopus [ 10 , 31 ]. However, since GS is a free database [ 56 ], it may be the only possible way to conduct a study. However, knowing that all databases are likely to contain errors, verifying a sample of data is a useful precaution.…”