Valid and Reliable? Basic and Expanded Recommendations for Psychometric Reporting and Quality Assessment.

Johansson, Magnus; Preuter, Marit; Karlsson, Simon; Möllerberg, Marie-Louise; Svensson, Hanna; Melin, Jeanette

doi:10.31219/osf.io/3htzc

Cited by 8 publications

(3 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Targeting of items compared to respondents was reviewed by visualizing the distribution of person locations on the same scale as the distribution of item threshold locations and calculating mean and standard deviations for both distributions. Measurement invariance was assessed by analyzing potential differential item functioning (DIF) related to age and gender, with 0.5 logits as a DIF size cutoff value (27). Reliability was assessed by a test information function (TIF) curve and reporting the proportions of respondents located within the scale range where reliability was estimated at or above person separation index 0.7.…”

Section: Discussionmentioning

confidence: 99%

“…Reliability was assessed by a test information function (TIF) curve and reporting the proportions of respondents located within the scale range where reliability was estimated at or above person separation index 0.7. No point estimate of reliability is reported, since it is more relevant to report the TIF, as it is based on item properties rather than sample properties (27).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A Psychometric Evaluation of the Swedish Translation of the Perceived Stress Scale: A Rasch Analysis

Forsström

Johansson

2023

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Stress reflects physical and psychological reactions to imposing demands and is often measured using self-reports. A widely-used instrument is the Perceived Stress Scale (PSS), intended to capture more general aspects of stress. A Swedish translation of the PSS is available but has not previously been examined using modern test theory approaches, such as Rasch analysis. Methods: The current study used data from 793 university students to investigate the dimensionality of different version of the PSS (14, 10, and 4 items) as well as potential response patterns among the participants. Results: The current study demonstrates that the PSS-14 has two separate factors, divided between negatively worded items (stress) and positively worded items (self-efficacy), although with only the negative subscale exhibiting good reliability. Response patterns were analyzed using Differential Item Functioning analysis, which did not find an influence of gender on the perceived difficulty of the items, but for age regarding the positive subscale (items 6 and 9). The PSS-10 also demonstrated adequate reliability for the negative subscale, but the PSS-4 was not deemed suitable as a unidimensional scale. Conclusions: Based on the results, none of the versions of the PSS should be used by sum-scoring all of the items. Only the negative items from the PSS-14 or PSS-10 can be used as unidimensional scales to measure general aspects of stress. Content validity is discussed, questioning the relevance of anger and being upset when measuring more general aspects of stress. Finally, a table to convert the PSS-7 ordinal sum scores to interval level scores is provided.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

A Psychometric Evaluation of the Swedish Translation of the Perceived Stress Scale: A Rasch Analysis

Forsström

Johansson

2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Summarily, when one factor accounts for the test performance of a student in a test is referred to as a uni-dimensional model. Also, Models in which it is assumed that more than one trait is required to account for student test performance are referred to as multidimensional [13].…”

Section: Dimensionality Assumption Of Irtmentioning

confidence: 99%

Test Dimensionality and Conditional Independence of Undergraduate Physics Test Items of National Open University of Nigeria: Perspective from Latent Traits Model (Ltm) Package of R Language

Nadrah

2023

ijmst

View full text Add to dashboard Cite

Physics test items developed by the National Open University of Nigeria (NOUN) to measure test-takers' proficiency in Elementary Mechanics, Heat, and Properties of Matter (PHY101) are intrinsically multidimensional. However, we observed the current method used by the University to score test-takers proficiency in the course is premised on Classical Test Theory, which has been criticised in the literature for its limitations, and a measurement model for unidimensional tests. Consequently, using an inappropriate model to model test-takers responses to items would adversely affect their true proficiency in the course. Therefore, this study assessed the dimensionality and conditional independence of physics test items of NOUN using the ltm package in the R language. A non-experimental design of survey research type was adopted. Test-takers responses to the 35 PHY 101 multiple-choice items across NOUN study centres in the 36 States of Nigeria were retrieved from the Directorate of Examination of Assessment (DEA) and used for the study, with empirical reliability of 0.80. Nine hundred seventy-eight test-takers responses were captured and analysed with modified parallel analysis test and Yen Q3 statistics implemented in ltm package of R language software, version 4.0.2. Findings remarked that NOUN physics items had more than one predominant dimension to account for the observed performance of test-takers in the course. Also, residual correlations of ten item pairs severely violated the conditional independence benchmark of 0.20. We conclude that PHY 101 of NOUN is intrinsically multidimensional. Test-takers responses to the test items are due to their locations on multiple latent variables. We recommend that a psychometric unit be established within the DEA of the University to check the appropriateness of the test items developed by the faculty lecturers.

show abstract

Psychometric Evaluation of the Parental Overprotective Measure: Toward a Reliable and Practical Assessment Tool

Elfström,

Åhlen

2024

J Child Fam Stud

View full text Add to dashboard Cite

Parental overprotection is thought to be a risk factor for childhood anxiety disorders. The Parental Overprotection Measure (POM) is a self-assessment scale that has been used in many studies, however, no article has comprehensively explored its psychometric properties. The aim of this study was to evaluate the POM to secure reliable, replicable, and comparable results. Using an item-response theoretical framework, we aimed to address five basic criteria of psychometric quality: one-dimensionality, ordered response categories, invariance, targeting, and reliability. Furthermore, we aimed to identify and exclude any items that did not contribute significantly, to create a concise and practical scale. A total of 1092 parents of children aged 4–12 years were recruited through an online advertisement and completed an anonymous online survey during September and October 2020. Out of the 19 items of the original scale, eight were excluded due to unsatisfactory psychometric properties and/or because of not being judged by experts as harmonious with the concept of parental overprotection. The retained items constituted the POM-11, a brief scale with sound psychometric properties that has the potential for use in both research and clinical settings.

show abstract

Valid and Reliable? Basic and Expanded Recommendations for Psychometric Reporting and Quality Assessment.

Cited by 8 publications

References 72 publications

A Psychometric Evaluation of the Swedish Translation of the Perceived Stress Scale: A Rasch Analysis

A Psychometric Evaluation of the Swedish Translation of the Perceived Stress Scale: A Rasch Analysis

Test Dimensionality and Conditional Independence of Undergraduate Physics Test Items of National Open University of Nigeria: Perspective from Latent Traits Model (Ltm) Package of R Language

Psychometric Evaluation of the Parental Overprotective Measure: Toward a Reliable and Practical Assessment Tool

Contact Info

Product

Resources

About