Another brick in the wall: a new ranking of academic journals in Economics using FDH

García-Romero, Antonio; Santín, Daniel; Sicilia, Gabriela

doi:10.1007/s11192-016-1843-3

Cited by 11 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It was originally proposed by Thompson et al (1986), and Adolphson, Cornia, & Walters (1991), coined the term 'Benefit-of-the-Doubt' by Melyn & Moesen (1991) who applied the method to obtain an indicator of macroeconomic performance, and subsequently formalized under a DEA framework by Lovell and Pastor (1999). In 2008, this approach was suggested by the OECD as a method to construct composite indicators (OECD, 2008), spurring many applications in different fields: among others, in bibliometrics (García-Romero, Santín, & Sicilia, 2016), health care (Shwartz, Burgess, & Zhu, 2016), competitiveness (Li & Zhao, 2015), police effectiveness (Verschelde & Rogge, 2012), and energy (Zanella, Camanho, & Dias, 2015).…”

Section: 'Benefit-of-the-doubt' (Bod)mentioning

confidence: 99%

Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach

Witte

Schiltz

2018

European Journal of Operational Research

View full text Add to dashboard Cite

show abstract

Section: 'Benefit-of-the-doubt' (Bod)mentioning

confidence: 99%

Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach

Witte

Schiltz

2018

European Journal of Operational Research

View full text Add to dashboard Cite

show abstract

“…Liebowitz and Palmer (1984) observed substantial differences for individual journals when the used citation-based indicator was adjusted for journal size (in terms of total number of characters published by a journal): Economic Journal dropped from rank 7 to rank 23, Oxford Economic Papers from rank 12 to rank 42, and Review of Economics and Statistics from rank 5 to rank 16. Garcia-Romero et al (2016) observed that Journal of Economic Literature would rank 2 using the Free Disposal Hull method (Deprins et al 1984) but 13 using the KMS method (Kalaitzidakis et al 2011). It was further observed that Journal of Economic Growth would rank 6 and 26, whereas Economics and Politics would rank 11 and 45 using the same two methods.…”

Section: Introductionmentioning

confidence: 99%

“…It has been shown that also various citation-based approaches to journal rankings are correlated. For instance, Harzing and van der Wal (2009) and Garcia-Romero et al (2016) found rank correlations in the range 0.67-0.91, whereas Bornmann et al (2018) reported rank correlations in the range 0.89-0.99 for the journal indicators outlined in Google Scholar. Now, this seems to imply that the citation-based ranking approach would not be of any great importance.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty and the ranking of economics journals

Lyhagen

Ahlgren

2020

Scientometrics

View full text Add to dashboard Cite

Journal rankings often show significant changes compared to previous rankings. This gives rise to the question of how well estimated the rank of a journal is. In this contribution, we consider uncertainty in a ranking of economics journals. We use the invariant method of Pinski and Narin to rank the journals. We propose an uncertainty measure, which is based on a bootstrap approach. The measure is the average absolute change in rank, which we see as a reasonable uncertainty measure regarding rankings. We further calculate, based on the bootstrap method, 95% confidence interval for the observed values of the invariant method. We show that ranks of the highest, as well as the lowest, ranked journals are well estimated, while there is a high degree of uncertainty regarding the rank of many mid-ranked journals. The distribution of the underlying measure is useful for identifying groups of journals that are more or less of the same quality (from the point of view of the invariant measure). The journal with the highest observed value of the invariant measure, Journal of Political Economy, has the best performance and constitutes a singleton, whereas Quarterly Journal of Economics and Econometrica form the next group (there is a slight overlap between the two with respect to confidence intervals). The journals ranked between about 190–230 form another group in which there are no major quality differences between the journals, as the confidence intervals are overlapping.

show abstract

“…Certain alternatives were found to combine multiple bibliometrics in a type of composite score, such as free disposable hull [ 40 ] and standardized average index [ 41 ]. Free disposable hull is an aggregation of four citation metrics (JIF, AI, h-index, and discounted impact factor) while standardized average index utilizes two metrics (JIF and h-index).…”

Section: Discussionmentioning

confidence: 99%

“…Free disposable hull is an aggregation of four citation metrics (JIF, AI, h-index, and discounted impact factor) while standardized average index utilizes two metrics (JIF and h-index). The combination of multiple factors provides better journal ranking by diversifying the number of outputs considered within one metric; however, these combinations may not be the most effective for optimal accuracy [ 40 , 41 ]. Further analysis and research is needed to optimize metric combinations; however, these factors still utilize JIF and the reported disadvantages - although offset by other metrics - still apply.…”

Section: Discussionmentioning

confidence: 99%

Evaluating Journal Impact Factor: a systematic survey of the pros and cons, and overview of alternative measures

Mech

Ahmed

Tamale

et al. 2020

J. Venom. Anim. Toxins incl. Trop. Dis

View full text Add to dashboard Cite

Background: Journal Impact Factor (JIF) has several intrinsic flaws, which highlight its inability to adequately measure citation distributions or indicate journal quality. Despite these flaws, JIF is still widely used within the academic community, resulting in the propagation of potentially misleading information. A critical review of the usefulness of JIF is needed including an overview of the literature to identify viable alternative metrics. The objectives of this study are: (1) to assess the usefulness of JIF by compiling and comparing its advantages and disadvantages; (2) to record the differential uses of JIF within research environments; and (3) to summarize and compare viable alternative measures to JIF. Methods: Three separate literature search strategies using MEDLINE and Web of Science were completed to address the three study objectives. Each search was completed in accordance with PRISMA guidelines. Results were compiled in tabular format and analyzed based on reporting frequency. Results: For objective (1), 84 studies were included in qualitative analysis. It was found that the recorded advantages of JIF were outweighed by disadvantages (18 disadvantages vs. 9 advantages). For objective (2), 653 records were included in a qualitative analysis. JIF was found to be most commonly used in journal ranking (n = 653, 100%) and calculation of scientific research productivity (n = 367, 56.2%). For objective (3), 65 works were included in qualitative analysis. These articles revealed 45 alternatives, which includes 18 alternatives that improve on highly reported disadvantages of JIF. Purpose Assessing the usefulness of JIF Assessing differential uses of JIF Overview of alternative measures

show abstract

Another brick in the wall: a new ranking of academic journals in Economics using FDH

Cited by 11 publications

References 24 publications

Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach

Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach

Uncertainty and the ranking of economics journals

Evaluating Journal Impact Factor: a systematic survey of the pros and cons, and overview of alternative measures

Contact Info

Product

Resources

About