Why We Need Systematic Reviews and Meta-Analyses in the Testing and Assessment Literature

Iliescu, Dragoş; Rusu, Andrei; Greiff, Samuel; Fokkema, Marjolein; Scherer, Ronny

doi:10.1027/1015-5759/a000705

Cited by 9 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A recent call to action underlined the importance of new systematic reviews and meta-analytic studies in the psychological assessment sphere in which burnout measures were cited as an example (see Iliescu et al, 2022). In heeding this call, we aimed to review the measurement properties (i.e., validity and internal consistency) of the MBI-GS and the evidence and study quality behind the MBI-GS.…”

Section: Previous Systematic Reviews and Meta-analysesmentioning

confidence: 99%

Maslach Burnout Inventory – General Survey

De Beer,

van der Vaart,

Escaffi-Schwarz

et al. 2024

European Journal of Psychological Assessment

View full text Add to dashboard Cite

This study sought to investigate the measurement properties of a “gold standard” instrument for measuring burnout, the Maslach Burnout Inventory – General Survey (MBI-GS), with a systematic literature review and meta-analysis of studies that had as the primary aim its psychometric validation. The search spanned from January 1996 to December 2022 using the following databases: Web of Science, Scopus, PubMed, APA PsychINFO, ScienceDirect, and ProQuest Dissertations and Theses Global. Our search identified 35 eligible studies for inclusion in the systematic review. Of these, 17 were included in the meta-analysis. The meta-analysis of the original 16-item version studies supported a three-dimensional representation of burnout with modest internal consistencies. However, similar to the narrative findings, it also indicated the viability of a two-factor solution. Consequently, the structural validity of the MBI-GS remains unclear, and so does its cross-cultural validity. The criterion validity of the cynicism and personal efficacy scales also raised a few questions. Overall, the findings provided some support for the adequacy of the measurement properties of the MBI-GS as a research tool measuring exhaustion, cynicism, and professional efficacy. However, we also identified challenges and practices of which researchers should be cognisant and that they should consider in future burnout projects.

show abstract

Section: Previous Systematic Reviews and Meta-analysesmentioning

confidence: 99%

Maslach Burnout Inventory – General Survey

De Beer,

van der Vaart,

Escaffi-Schwarz

et al. 2024

European Journal of Psychological Assessment

View full text Add to dashboard Cite

show abstract

“…Since there are known shortcomings regarding measurement quality in academic dishonesty research, we provide an in-depth investigation of measurement practices to advance the field in particular and awareness of methodological best practices in psychology in general (Flake & Fried, 2020). Such systematic reviews on assessment measures are also particularly vital for the advancement of the field of Psychological Assessment, because they contribute to a comprehensive understanding of measurement properties, rigorous assessment, cross-cultural applicability, and practical guidance (Iliescu et al, 2022).…”

Section: The Current Researchmentioning

confidence: 99%

“…This will support methodological rigor and improve consistency of future research, which will increase the field's productivity as a whole. To this end, specific systematic reviews on the assessment methods are necessary (Iliescu et al, 2022). It is notable that despite their strengths and trendiness, such reviews are still used rather infrequently in the psychological assessment literature.…”

Section: Introductionmentioning

confidence: 99%

How to best measure academic dishonesty in students: A systematic review of self-report assessment methods and psychometric quality - Preprint

Fritz,

Cruz,

Janke

et al. 2023

Preprint

View full text Add to dashboard Cite

Addressing a pervasive problem in educational institutions, investigation into academic dishonesty by students has produced a vast body of empirical research, mostly based on self-report measures. However, the literature repeatedly points to inconsistencies in assessment methods and unclear measurement quality. We conducted a preregistered systematic review to provide a comprehensive overview of self-report assessments (including past cheating behavior and cheating intentions), and to evaluate operationalizations and psychometric quality. In 256 instruments of 231 studies, 70% were unique self-report measures, meaning newly created or adapted instruments, and 38.7% of studies did not investigate or report psychometric quality criteria. As such, the academic dishonesty literature is no exception in questionable measurement practices (Flake & Fried, 2020), where reports often lack key measurement information, such as the full item list (missing in 32%) or the time frames for behavioral assessment (missing in 50.4%). Our findings also illustrate how a lack of psychometric understanding of the construct and inconsistencies regarding the spectrum of academic cheating behaviors (Simha & Cullen, 2012) threatens the interpretability and comparability of research findings. Our findings can assist researchers in the selection of suitable measures to assess academic dishonesty, and thereby raise the comparability, replicability and validity of study results.

show abstract

“…At first glance, meta-analyses meet the criteria of combining different single studies, therefore different perspectives. It is no coincidence that they are hyped in the current scientific milieu (e.g., Iliescu et al, 2022). Freese and Peterson (2018) argue that single study results often are influenced by scientists' "scientific selves, " which, in turn, are affected by different interests, such as emotional or economic.…”

Section: Collective Objectivity In a Statistical Understandingmentioning

confidence: 99%

Future Objectivity Requires Perspective and Forward Combinatorial Meta-Analyses

Hanfstingl

2022

Front. Psychol.

View full text Add to dashboard Cite

This manuscript contributes to a future definition of objectivity by bringing together recent statements in epistemology and methodology. It outlines how improved objectivity can be achieved by systematically incorporating multiple perspectives, thereby improving the validity of science. The more result-biasing perspectives are known, the more a phenomenon of interest can be disentangled from these perspectives. Approaches that call for the integration of perspective into objectivity at the epistemological level or that systematically incorporate different perspectives at the statistical level already exist and are brought together in the manuscript. Recent developments in research methodology, such as transparency, reproducibility of research processes, pre-registration of studies, or free access to raw data, analysis strategies, and syntax, promote the explication of perspectives because they make the entire research process visible. How the explication of perspectives can be done practically is outlined in the manuscript. As a result, future research programs can be organized in such a way that meta-analyses and meta-meta-analyses can be conducted not only backward but forward and prospectively as a regular and thus well-prepared part of objectification and validation processes.

show abstract

Why We Need Systematic Reviews and Meta-Analyses in the Testing and Assessment Literature

Cited by 9 publications

References 36 publications

Maslach Burnout Inventory – General Survey

Maslach Burnout Inventory – General Survey

How to best measure academic dishonesty in students: A systematic review of self-report assessment methods and psychometric quality - Preprint

Future Objectivity Requires Perspective and Forward Combinatorial Meta-Analyses

Contact Info

Product

Resources

About