Toward More Inclusive Metrics and Open Science to Measure Research Assessment in Earth and Natural Sciences

Pourret, Olivier; Irawan, Dasapta Erwin; Shaghaei, Najmeh; Rijsingen, Elenora M. van; Besançon, Lonni

doi:10.3389/frma.2022.850333

Cited by 4 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Majority of included studies were in the field of health sciences (n = 10), followed by biomedicine/and or health sciences and multidisciplinary, (n = 7 and n = 6, respectively). One study 32 proposed "open access for measuring research assessment in Earth and Natural Sciences". Several studies targeted their initiatives mainly towards redesigning or modifying researcher assessments via new metrics, measures, methods, and incentives.…”

Section: Arɵcled Excluded (A Discussion About Challenges and Opportun...mentioning

confidence: 99%

Open science interventions proposed or implemented to assess researcher impact: a scoping review

Ghannad,

Armond,

et al. 2023

F1000Res

View full text Add to dashboard Cite

Background Several open science-promoting initiatives have been proposed to improve the quality of biomedical research, including initiatives for assessing researchers’ open science behaviour as criteria for promotion or tenure. Yet there is limited evidence to judge whether the interventions are effective. This review aimed to summarise the literature, identifying open science practices related to researcher assessment, and map the extent of evidence of existing interventions implemented to assess researchers and research impact. Methods A scoping review using the Joanna Briggs Institute Scoping Review Methodology was conducted. We included all study types that described any open science practice-promoting initiatives proposed or implemented to assess researchers and research impact, in health sciences, biomedicine, psychology, and economics. Data synthesis was quantitative and descriptive. Results Among 18,020 identified documents, 27 articles were selectedfor analysis. Most of the publications were in the field of health sciences (n = 10), and were indicated as research culture, perspective, commentary, essay, proceedings of a workshop, research article, world view, opinion, research note, editorial, report, and research policy articles (n = 22). The majority of studies proposed recommendations to address problems regarding threats to research rigour and reproducibility that were multi-modal (n = 20), targeting several open science practices. Some of the studies based their proposed recommendations on further evaluation or extension of previous initiatives. Most of the articles (n = 20) did not discuss implementation of their proposed intervention. Of the 27 included articles, 10 were cited in policy documents, with The Leiden Manifesto being the most cited (104 citations). Conclusion This review provides an overview of proposals to integrate open science into researcher assessment. The more promising ones need evaluation and, where appropriate, implementation. Study registration https://osf.io/ty9m7

show abstract

Section: Arɵcled Excluded (A Discussion About Challenges and Opportun...mentioning

confidence: 99%

Open science interventions proposed or implemented to assess researcher impact: a scoping review

Ghannad,

Armond,

et al. 2023

F1000Res

View full text Add to dashboard Cite

show abstract

“…Within the wide context of scientific topics, such as science of science, our community has undertaken commendable efforts in analyzing gender inequality (Jackson et al, 2022 ) and crafting inclusive metrics (Pourret et al, 2022 ). However, a significant practical challenge lies in scaling up these analyses from specific regions, individual disciplines, and singular data sources to comprehensive global studies with integrated knowledge graphs.…”

Section: Challenge 3: Inclusivenessmentioning

confidence: 99%

Responsible models and indicators: challenges from artificial intelligence

Zhang

2023

Front. Res. Metr. Anal.

View full text Add to dashboard Cite

Are Open Science instructions targeted to ecologists and evolutionary biologists sufficient? A literature review of guidelines and journal data policies

Koivisto,

Mäntylä

2024

Ecology and Evolution

View full text Add to dashboard Cite

Open science (OS) awareness and skills are increasingly becoming an essential part of everyday scientific work as e.g., many journals require authors to share data. However, following an OS workflow can seem challenging at first. Thus, instructions by journals and other guidelines are important. But how comprehensive are they in the field of ecology and evolutionary biology (Ecol Evol)? To find this out, we reviewed 20 published OS guideline articles aimed for ecologists or evolutionary biologists, together with the data policies of 17 Ecol Evol journals to chart the current landscape of OS guidelines in the field, find potential gaps, identify field‐specific barriers for OS and discuss solutions to overcome these challenges. We found that many of the guideline articles covered similar topics, despite being written for a narrow field or specific target audience. Likewise, many of the guideline articles mentioned similar obstacles that could hinder or postpone a transition to open data sharing. Thus, there could be a need for a more widely known, general OS guideline for Ecol Evol. Following the same guideline could also enhance the uniformity of the OS practices carried on in the field. However, some topics, like long‐term experiments and physical samples, were mentioned surprisingly seldom, although they are typical issues in Ecol Evol. Of the journals, 15 out of 17 expected or at least encouraged data sharing either for all articles or under specific conditions, e.g. for registered reports and 10 of those required data sharing at the submission phase. The coverage of journal data policies varied greatly between journals, from practically non‐existing to very extensive. As journals can contribute greatly by leading the way and making open data useful, we recommend that the publishers and journals would invest in clear and comprehensive data policies and instructions for authors.

show abstract

Toward More Inclusive Metrics and Open Science to Measure Research Assessment in Earth and Natural Sciences

Cited by 4 publications

References 33 publications

Open science interventions proposed or implemented to assess researcher impact: a scoping review

Open science interventions proposed or implemented to assess researcher impact: a scoping review

Responsible models and indicators: challenges from artificial intelligence

Are Open Science instructions targeted to ecologists and evolutionary biologists sufficient? A literature review of guidelines and journal data policies

Contact Info

Product

Resources

About