Evaluation of the National Institutes of Health–supported relative citation ratio among American orthopedic spine surgery faculty: A new bibliometric measure of scientific influence

Dijanic, Christopher; Sudah, Suleiman Y.; Michel, C; Smith, Tiffany A.; Patankar, Aneesh; Manzi, Joseph E.; Faccone, Robert D.; Kerrigan, Daniel; Menendez, Mariano E.; Cohen, Jason

doi:10.1016/j.xnsj.2022.100143

Cited by 4 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, RCR analyses have only been performed in limited academic fields. 7,10,13–23 This study expands on the utilization of the RCR to the subspecialty of academic cornea and external diseases and provides benchmark data for the field. Our analysis demonstrates that academic cornea and external diseases ophthalmologists produce highly impactful research, as highlighted by the high median RCR value (1.40) relative to the NIH standard RCR value of 1.0.…”

Section: Discussionmentioning

confidence: 98%

Evaluation of Research Productivity Among Academic Cornea, External Diseases, and Refractive Surgery Ophthalmologists Using the Relative Citation Ratio

Guan,

Henderson,

Singh

et al. 2024

Cornea

View full text Add to dashboard Cite

Purpose: The purpose of this study was to provide relative citation ratio (RCR) benchmark data for cornea and external diseases specialists. Design: This is a cross-sectional bibliometric analysis. Subjects: Subjects included were fellowship-trained cornea and external diseases faculty at Accreditation Council for Graduate Medical Education–accredited institutions in the United States. Methods: Academic specialists were indexed using the National Institutes of Health iCite Web site. Publication count, mean RCR score, and weighted RCR score were obtained between October 2022 and January 2023 by examining PubMed-listed publications from 1980 to 2022. Data were compared by sex, career duration, academic rank, and acquisition of a Doctor of Philosophy. Main outcome measures: The main outcome measures were publication count, mean RCR value, and weighted RCR value. Results: The cohort included 602 specialists from 112 Accreditation Council for Graduate Medical Education–accredited institutions. These clinician–scientists produced highly impactful research with a median publication count of 15 (interquartile ranges 4–41), median RCR of 1.4 (interquartile ranges 0.91–1.88), and median-weighted RCR of 20.28 (5.3–66.69). Both academic rank and career length were associated with greater publication count and RCR values. Male sex was also associated with greater publications counts and RCR scores compared with female faculty. Acquisition of a Doctor of Philosophy was associated with greater publication counts and weighted RCR scores but no difference in mean RCR scores. Conclusions: Academic cornea and external diseases specialists conduct high-impact research, with a median RCR of 1.4, exceeding the NIH standard value of 1. These data provide RCR benchmark data for the field to inform self, institutional, and departmental evaluations. These results also highlight a significant gender disparity in the field necessitating efforts to increase female representation and ensure equal opportunities.

show abstract

Section: Discussionmentioning

confidence: 98%

Evaluation of Research Productivity Among Academic Cornea, External Diseases, and Refractive Surgery Ophthalmologists Using the Relative Citation Ratio

Guan,

Henderson,

Singh

et al. 2024

Cornea

View full text Add to dashboard Cite

show abstract

“…20,25 RCR utilization, rather than h-index use, is increasing in other fields of academic medicine, including neurosurgery, cardiothoracic surgery, plastic surgery, and spine surgery. [26][27][28][29] For instance, one study compared RCR with h-index in academic ophthalmology and how these measures were associated with sociodemographic factors and outcomes such as academic rank and PhD or fellowship attainment. 19,24 The findings suggested that RCR, both mean and weighted, is a more effective measure of research productivity than the h-index.…”

Section: Discussionmentioning

confidence: 99%

“…H‐index is also not normalized across fields, meaning that it is mostly useful for comparison of individuals within the same specialty 20,25 . RCR utilization, rather than h‐index use, is increasing in other fields of academic medicine, including neurosurgery, cardiothoracic surgery, plastic surgery, and spine surgery 26–29 . For instance, one study compared RCR with h‐index in academic ophthalmology and how these measures were associated with sociodemographic factors and outcomes such as academic rank and PhD or fellowship attainment 19,24 .…”

Section: Discussionmentioning

confidence: 99%

The Relative Citation Ratio: An Impartial Assessment of Productivity in Academic Otolaryngologists

et al. 2023

View full text Add to dashboard Cite

ObjectivesQuantification of academic productivity relies on bibliometric measurements, such as the Hirsch index (h‐index). The National Institutes of Health (NIH) recently developed the relative citation ratio (RCR), an article‐level, citation‐driven metric that compares researchers with others within their respective fields. Our study is the first to compare the usage of RCR in academic otolaryngology.Study DesignRetrospective Database Review.MethodsAcademic otolaryngology residency programs were identified using the 2022 Fellowship and Residency Electronic Interactive Database. Demographic and training data were collected for surgeons using institutional websites. RCR was calculated using the NIH iCite tool, and h‐index was calculated using Scopus. Mean RCR (m‐RCR) is the average score of the author's articles. Weighted RCR (w‐RCR) is the sum of all article scores. These derivatives are a measure of impact and output, respectively. The career duration of a physician was categorized into the following cohorts: 0–10, 11–20, 21–30, and 31+ years.ResultsA total of 1949 academic otolaryngologists were identified. Men had higher h‐indices and w‐RCRs than women (both p less than 0.001). m‐RCR was not different between genders (p = 0.083). There was a difference in h‐index and w‐RCR (both p less than 0.001) among the career duration cohorts, but there was no difference in m‐RCR among the cohorts (p = 0.416). The faculty rank professor was the greatest for all metrics (p < 0.001).ConclusionCritics of the h‐index argue that it is reflective of the time a researcher has spent in the field, instead of impact. The RCR may reduce historic bias against women and younger otolaryngologists.Level of EvidenceN/A Laryngoscope, 2023

show abstract

“…The greater appropriateness of RCR over the JIF for the analysis of the impact of an article in the community of interest has been proposed by Surkis and Spore (2018). The mean and weighted RCR values (the total RCR scores of the group of articles under consideration) have also been suggested as appropriate tool to compare research productivity in the academic environment in different medical fields (Reddy et al, 2020;Dijanic et al, 2022;Patel and Ali, 2022). The analysis herein presented indicated that the iCite tools could be adopted by (major and minor) funding organizations other than the NIH to evaluate their research investment, by using NIH funding as a reference.…”

Section: Assessment Of Arisla-derived Researchmentioning

confidence: 99%

“…The Web of Science Core Collection TM by Clarivate TM provided a list of peer-reviewed publications acknowledging AriSLA funds, as well as their level of influence by the scientific community in the Web of Science Citation Report. In addition, a bibliometric international benchmark analysis was conducted using the relative citation ratio (RCR) metric provided by the iCite platform of the NIH Office of Portfolio Analysis (Santangelo, 2017; iCite | New Analysis | NIH Office of Portfolio Analysis ) This tool is adopted by the NIH and other main research institutes, particularly in the United States for the analysis of their investment (Santangelo, 2017;Surkis and Spore, 2018;Rechtman et al, 2022), or by learned societies for comparison of research productivity amongst academic faculty (Reddy et al, 2020;Dijanic et al, 2022;Patel and Ali, 2022). However, its use by funding agencies and charities for international benchmark is still rather limited, or at least this information is not easily accessible.…”

Section: Introductionmentioning

confidence: 99%

The positive impact on translational research of Fondazione italiana di ricerca per la Sclerosi Laterale Amiotrofica (AriSLA), a non-profit foundation focused on amyotrophic lateral sclerosis. Convergence of ex-ante evaluation and ex-post outcomes when goals are set upfront

Guareschi,

Ravasi,

Baldessari

et al. 2023

Front. Res. Metr. Anal.

View full text Add to dashboard Cite

Charities investing on rare disease research greatly contribute to generate ground-breaking knowledge with the clear goal of finding a cure for their condition of interest. Although the amount of their investments may be relatively small compared to major funders, the advocacy groups' clear mission promotes innovative research and aggregates highly motivated and mission-oriented scientists. Here, we illustrate the case of Fondazione italiana di ricerca per la Sclerosi Laterale Amiotrofica (AriSLA), the main Italian funding agency entirely dedicated to amyotrophic lateral sclerosis research. An international benchmark analysis of publications derived from AriSLA-funded projects indicated that their mean relative citation ratio values (iCite dashboard, National Institutes of Health, U.S.) were very high, suggesting a strong influence on the referring international scientific community. An interesting trend of research toward translation based on the “triangle of biomedicine” and paper citations (iCite) was also observed. Qualitative analysis on researchers' accomplishments was convergent with the bibliometric data, indicating a high level of performance of several working groups, lines of research that speak of progression toward clinical translation, and one study that has progressed from the investigation of cellular mechanisms to a Phase 2 international clinical trial. The key elements of the success of the AriSLA investment lie in: (i) the clear definition of the objectives (research with potential impact on patients, no matter how far), (ii) a rigorous peer-review process entrusted to an international panel of experts, (iii) diversification of the portfolio with ad hoc selection criteria, which also contributed to bringing new experts and younger scientists to the field, and (iv) a close interaction of AriSLA stakeholders with scientists, who developed a strong sense of belonging. Periodic review of the portfolio of investments is a vital practice for funding agencies. Sharing information between funding agencies about their own policies and research assessment methods and outcomes help guide the international debate on funding strategies and research directions to be undertaken, particularly in the field of rare diseases, where synergy is a relevant enabling factor.

show abstract

Evaluation of the National Institutes of Health–supported relative citation ratio among American orthopedic spine surgery faculty: A new bibliometric measure of scientific influence

Cited by 4 publications

References 18 publications

Evaluation of Research Productivity Among Academic Cornea, External Diseases, and Refractive Surgery Ophthalmologists Using the Relative Citation Ratio

Evaluation of Research Productivity Among Academic Cornea, External Diseases, and Refractive Surgery Ophthalmologists Using the Relative Citation Ratio

The Relative Citation Ratio: An Impartial Assessment of Productivity in Academic Otolaryngologists

The positive impact on translational research of Fondazione italiana di ricerca per la Sclerosi Laterale Amiotrofica (AriSLA), a non-profit foundation focused on amyotrophic lateral sclerosis. Convergence of ex-ante evaluation and ex-post outcomes when goals are set upfront

Contact Info

Product

Resources

About