Investigating the Use of Written and Performance-Based Testing to Summarize Competence on the Case Management Component of the NBCE Part IV–National Practical Examination

Townsend, Paul D.; Christensen, Mark G.; Kreiter, Clarence D.; zumBrunnen, James R.

doi:10.1080/10401330903445737

Cited by 2 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scores based solely on performance-based stations, such as the MMI require extended testing time to achieve acceptable generalizability, to which would be added time for question development and training. Combining scores from performance-based formats and written formats may improve test generalizability, and methods to do this already exist [42,43]. It could be possible that the combined NAC score was more generalizable than either of the two measures individually, and potentially a better use of resources.…”

Section: Discussionmentioning

confidence: 99%

The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training

et al. 2014

View full text Add to dashboard Cite

BackgroundEntry into specialty training was determined by a National Assessment Centre (NAC) approach using a combination of a behavioural Multiple-Mini-Interview (MMI) and a written Situational Judgement Test (SJT). We wanted to know if interviewers could make reliable and valid decisions about the non-cognitive characteristics of candidates with the purpose of selecting them into general practice specialty training using the MMI. Second, we explored the concurrent validity of the MMI with the SJT.MethodsA variance components analysis estimated the reliability and sources of measurement error. Further modelling estimated the optimal configurations for future MMI iterations. We calculated the relationship of the MMI with the SJT.ResultsData were available from 1382 candidates, 254 interviewers, six MMI questions, five alternate forms of a 50-item SJT, and 11 assessment centres. For a single MMI question and one assessor, 28% of the variance between scores was due to candidate-to-candidate variation. Interviewer subjectivity, in particular the varying views that interviewer had for particular candidates accounted for 40% of the variance in scores. The generalisability co-efficient for a six question MMI was 0.7; to achieve 0.8 would require ten questions. A disattenuated correlation with the SJT (r = 0.35), and in particular a raw score correlation with the subdomain related to clinical knowledge (r = 0.25) demonstrated evidence for construct and concurrent validity. Less than two per cent of candidates would have failed the MMI.ConclusionThe MMI is a moderately reliable method of assessment in the context of a National Assessment Centre approach. The largest source of error relates to aspects of interviewer subjectivity, suggesting enhanced interviewer training would be beneficial. MMIs need to be sufficiently long for precise comparison for ranking purposes. In order to justify long term sustainable use of the MMI in a postgraduate assessment centre approach, more theoretical work is required to understand how written and performance based test of non-cognitive attributes can be combined, in a way that achieves acceptable generalizability, and has validity.

show abstract

Section: Discussionmentioning

confidence: 99%

The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training

et al. 2014

View full text Add to dashboard Cite

show abstract

“…In January 1996, the NBCE introduced the Part IV exam, the large-scale assessment for chiropractors, which was made up of 3 domains: diagnostic imaging, chiropractic technique, and case management. 29 The main component of Part IV is the objective structured clinical examination (OSCE), an assessment designed to test clinical skill performance and competency by simulating real-world procedures. 30 The OSCE is a standard mode of assessment of medical competency and clinical skills in the United States, Canada, and the United Kingdom.…”

Section: Future Directions For the Part IV Practical Examinationmentioning

confidence: 99%

Current state and future directions of the National Board of Chiropractic Examiners

Ouzts¹,

Himelfarb²,

Shotts³

et al. 2020

Journal of Chiropractic Education

View full text Add to dashboard Cite

The objective of this paper is to describe changes made to chiropractic national board examinations in the United States, including methodologies in test scoring, and to discuss future directions in test development and administration being considered by the National Board of Chiropractic Examiners (NBCE). Additionally, this paper serves as an introduction to the articles written by the NBCE staff and published in this issue of the journal. Statistical perspective on the properties of a test are presented, and reasons for the NBCE moving to item response theory for test scoring are described. NBCE consideration of on-demand testing and changes implemented in the Part IV practical examination are also discussed.

show abstract

Investigating the Use of Written and Performance-Based Testing to Summarize Competence on the Case Management Component of the NBCE Part IV–National Practical Examination

Abstract: These results provide guidance for improving the generalizability of the standardized patient-based stations by combining the scores on the performance and written components using appropriate weighting procedures.

Cited by 2 publications

References 15 publications

The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training

The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training

Current state and future directions of the National Board of Chiropractic Examiners

Contact Info

Product

Resources

About