Bibliometrics: The Leiden Manifesto for research metrics

Hicks, Diana; Wouters, Paul; Waltman, Ludo; Rijcke, Sarah de; Ràfols, Ismael

doi:10.1038/520429a

Cited by 1,865 publications

(1,471 citation statements)

References 10 publications

Supporting

Mentioning

1,131

Contrasting

Unclassified

182

Order By: Relevance

“…Much has been written about how the pursuit of flashy papers can push scientists to crowd into similar, competitive projects, cut corners or exaggerate the significance of their findings [4][5][6] . We think that the problem is more profound: popular short-term bibliometric measures discourage the kind of risky research that is likely to shift the knowledge frontier 7 .…”

Section: Research Impactmentioning

confidence: 99%

“…The review panels we observed last year were using bibliometrics in much the same way as they did before the 2015 Leiden Manifesto 4 , the 2012 San Francisco Declaration on Research Assessment, which Nature is signing (see page 394), and similar exhortations against their use. After all, bibliometric measures offer a convenient way to help evaluate a large number of proposals and papers.…”

mentioning

confidence: 90%

“…As participants on multiple review panels and scientific councils, we have heard many lament researchers' reluctance to take risks. Yet we've seen the same panels eschew risk and rely on bibliometric indicators for assessments, despite widespread agreement that they are imperfect measures [1][2][3][4][5][6] .…”

mentioning

confidence: 99%

See 2 more Smart Citations

Reviewers are blinkered by bibliometrics

Stephan

Veugelers

Wang

2017

Nature

157

View full text Add to dashboard Cite

Section: Research Impactmentioning

confidence: 99%

mentioning

confidence: 90%

See 1 more Smart Citation

Reviewers are blinkered by bibliometrics

Stephan

Veugelers

Wang

2017

Nature

157

View full text Add to dashboard Cite

“…First, the statistical nature of data in the citation analysis does not satisfy the standard requirements for a safe application of the usual methods. Second, their definitions and the procedure for computing them are too complex to allow an easy-to-handle analysis, which contradicts one of the main requirements for being a good bibliometric index (see [22]): the meaning of such an index must be more or less evident from the mathematical point of view in order to allow a direct bibliometric analysis.…”

Section: Methodsmentioning

confidence: 99%

“…The reader can find a great deal of literature related to this topic in specialized journals (see [9][10][11][12][13][14][15][16][17][18]). Additionally, some institutions and groups of research analysts have presented reports and conclusions about this topic-generally in the direction of showing the negative effects of the automatic use of bibliometric indicators in research evaluation; see for example the IMU report, San Francisco Declaration on Research Assessment (DORA), the Metric Tide 2015, and the Leiden Manifesto [19][20][21][22]. However, it must be said that some researchers consider that these criticisms on the metric-based systems must be analyzed in national contexts, since the usage of other procedures-such as peer-review-based systems-also has limitations.…”

Section: Introductionmentioning

confidence: 99%

Measuring Time-Dynamics and Time-Stability of Journal Rankings in Mathematics and Physics by Means of Fractional p-Variations

2017

View full text Add to dashboard Cite

Journal rankings of specific research fields are often used for evaluation purposes, both of authors and institutions. These rankings can be defined by means of several methods, as expert assessment, scholarly-based agreements, or by the ordering induced by a numeric index associated to the prestige of the journals. In order to be efficient and accepted by the research community, it must preserve the ordering over time, at least up to a point. Otherwise, the procedure for defining the ranking must be revised to assure that it reflects the presumably stable characteristic "prestige" that it claims to be quantifying. A mathematical model based on fractional p-variations of the values of the order number of each journal in a time series of journal rankings is explained, and its main properties are shown. As an example, we study the evolution of two given ordered lists of journals through an eleven-year series. These journal ranks are defined by using the 2-year Impact Factor of Thomson-Reuters (nowadays Clarivate Analytics) lists for MATHEMATICS and PHYSICS, APPLIED from 2002 to 2013. As an application of our model, we define an index that precludes the use of journal ranks for evaluation purposes when some minimal requirements on the associated fractional p-variations are not satisfied. The final conclusion is that the list of mathematics does not satisfy the requirements on the p-variations, while the list of applied physics does.

show abstract

Federal Funding and Citation Metrics of US Biomedical Researchers, 1996 to 2022

2022

View full text Add to dashboard Cite

ImportanceBoth citation and funding metrics converge in shaping current perceptions of academic success.ObjectiveTo evaluate what proportion of the most-cited US-based scientists are funded by biomedical federal agencies and whether funded scientists are more cited than nonfunded ones.Design, Setting, and ParticipantsThis survey study used linkage of a Scopus-based database on top-cited US researchers (according to a composite citation metric) and the National Institutes of Health RePORTER database of federal funding (33 biomedical federal agencies). Matching was based on name and institution. US-based top-cited scientists who were allocated to any of 69 scientific subfields highly related to biomedicine were considered in the main analysis. Data were downloaded on June 11, 2022.Main Outcomes and MeasuresProportion of US-based top-cited biomedical scientists who had any (1996-2022), recent (2015-2022), and current (2021-2022) funding. Comparisons of funded and nonfunded scientists assessed total citations and a composite citation index.ResultsThere were 204 603 records in RePORTER (1996-2022) and 75 316 US-based top-cited scientists in the career-long citation database; 40 887 scientists were included in the main analysis. The proportion of US-based top-cited biomedical scientists (according to career-long citation impact) who had received any federal funding from biomedical research agencies was 62.7% (25 650 of 40 887) for any funding (1996-2022), 23.1% (9427 of 40 887) for recent funding (2015-2022), and 14.1% (5778 of 40 887) for current funding (2021-2022). Respective proportions were 64.8%, 31.4%, and 20.9%, for top-cited scientists according to recent single-year citation impact. There was large variability across scientific subfields (eg, current funding: 31% of career-long impact top-cited scientists in geriatrics, 30% in bioinformatics and 29% in developmental biology, but 0% in legal and forensic medicine, general psychology and cognitive sciences, and gender studies). Funded top-cited researchers were overall more cited than nonfunded top-cited scientists (median [IQR], 9594 [5650-1703] vs 5352 [3057-9890] citations; P &lt; .001) and substantial difference remained after adjusting for subfield and years since first publication. Differences were more prominent in some specific biomedical subfields.Conclusions and RelevanceIn this survey study, biomedical federal funding had offered support to approximately two-thirds of the top-cited biomedical scientists at some point during the last quarter century, but only a small minority of top-cited scientists had current federal biomedical funding. The large unevenness across subfields needs to be addressed with ways that improve equity, efficiency, excellence, and translational potential.

show abstract

Bibliometrics: The Leiden Manifesto for research metrics

Cited by 1,865 publications

References 10 publications

Reviewers are blinkered by bibliometrics

Reviewers are blinkered by bibliometrics

Measuring Time-Dynamics and Time-Stability of Journal Rankings in Mathematics and Physics by Means of Fractional p-Variations

Federal Funding and Citation Metrics of US Biomedical Researchers, 1996 to 2022

Contact Info

Product

Resources

About