The Presence and Impact of Local Item Dependence on Objective Structured Clinical Examinations Scores and the Potential Use of the Polytomous, Many-Facet Rasch Model

Lawson, Douglas M.; Brailovsky, Carlos

doi:10.1016/j.jmpt.2006.08.002

Cited by 6 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MFRM ([12], see [19-22] for applications) is a formal model for transforming nonlinear, scale-dependent ordinal raw scores into linear, scale-free interval measures. In its basic form, the MFRM represents the probability P nijk of a resident n being given by judge j a score k on an item i as an effect of the ability of resident n (β n ), the difficulty of item i (δ i ), the severity of judge j (γ j ), and the impediment in giving a score k rather than k -1 (τ k ):…”

Section: Methodsmentioning

confidence: 99%

A short questionnaire to assess pediatric resident’s competencies: the validation process

et al. 2013

View full text Add to dashboard Cite

BackgroundIn order to help assess resident performance during training, the Residency Affair Committee of the Pediatric Residency Program of the University of Padua (Italy) administered a Resident Assessment Questionnaire (ReAQ), which both residents and faculty were asked to complete. The aim of this article is to present the ReAQ and its validation.MethodsThe ReAQ consists of 20 items that assess the six core competencies identified by the Accreditation Council of Graduate Medical Education (ACGME). A many-facet Rasch measurement analysis was used for validating the ReAQ.ResultsBetween July 2011 and June 2012, 211 evaluations were collected from residents and faculty. Two items were removed because their functioning changed with the gender of respondents. The step calibrations were ordered. The self evaluations (residents rating themselves) positively correlated with the hetero evaluations (faculty rating residents; Spearman’s ρ = 0.75, p < 0.001). Unfortunately, the observed agreement among faculty was smaller than expected (Exp = 47.1%; Obs = 41%), which indicates that no enough training to faculty for using the tool was provided.ConclusionsIn its final form, the ReAQ provides a valid unidimensional measure of core competences in pediatric residents. It produces reliable measures, distinguishes among groups of residents according to different levels of performance, and provides a resident evaluation that holds an analogous meaning for residents and faculty.

show abstract

Section: Methodsmentioning

confidence: 99%

A short questionnaire to assess pediatric resident’s competencies: the validation process

et al. 2013

View full text Add to dashboard Cite

show abstract

“…More advanced techniques such as item response theory (IRT) employing the many facet Rasch model (MFRM) have been employed for assessing the independence of items in chiropractic and nursing OSCEs [17]. This study, however, employed only a single parameter (item difficulty) in the application of IRT and did not study the temporal stability of the OSCE.…”

Section: Introductionmentioning

confidence: 99%

Temporal stability of objective structured clinical exams: a longitudinal study employing item response theory

Baig

Violato

2012

BMC Med Educ

View full text Add to dashboard Cite

BackgroundThe objective structure clinical examination (OSCE) has been used since the early 1970s for assessing clinical competence. There are very few studies that have examined the psychometric stability of the stations that are used repeatedly with different samples. The purpose of the present study was to assess the stability of objective structured clinical exams (OSCEs) employing the same stations used over time but with a different sample of candidates, SPs, and examiners.MethodsAt Time 1, 191 candidates and at Time 2 (one year apart), 236 candidates participated in a 10-station OSCE; 6 of the same stations were used in both years. Generalizability analyses (Ep2) were conducted. Employing item response analyses, test characteristic curves (TCC) were derived for each of the 6 stations for a 2-parameter model. The TCCs were compared across the two years, Time 1 and 2.ResultsThe Ep2 of the OSCEs exceeded.70. Standardized thetas (θ) and discriminations were equivalent for the same station across the two year period indicating equivalent TCCs for a 2-parameter model.ConclusionThe 6 OSCE stations used by the AIMG program over two years have adequate internal consistency reliability, stable generalizability (Ep2) and equivalent test characteristics. The process of assessment employed for IMG’s are stable OSCE stations that may be used several times over without compromising psychometric properties.With careful security, high-stakes OSCEs may use the same stations that have high internal consistency and generalizability repeatedly as the psychometric properties are stable over several years with different samples of candidates.

show abstract

Local Dependence

Marais¹

2012

Rasch Models in Health

View full text Add to dashboard Cite

The Presence and Impact of Local Item Dependence on Objective Structured Clinical Examinations Scores and the Potential Use of the Polytomous, Many-Facet Rasch Model

Cited by 6 publications

References 19 publications

A short questionnaire to assess pediatric resident’s competencies: the validation process

A short questionnaire to assess pediatric resident’s competencies: the validation process

Temporal stability of objective structured clinical exams: a longitudinal study employing item response theory

Local Dependence

Contact Info

Product

Resources

About