Reliability and construct validity of some cognitive measures of clinical reasoning

Abstract

The past few years have seen a growing interest in understanding clinical reasoning processes by using the research methods of cognitive psychology. Nevertheless, a gap remains between the basic research methods and the application of these methods as evaluative instruments. The present research applied standard psychometric "yardsticks"-reliability and construct validity-to several cognitive measures of recall and perception used in previous studies. The cognitive measures were compared to measures of clinical performance derived from encounters with simulated patients and to conventional measures of knowledge based on multiple-choice questions and modified essay questions.The cognitive measures demonstrated reliability comparable to that of the conventional tests and considerably exceeding that of the clinical performance measures. Validity coefficients were comparable for all the measures. A calculation of reliability per unit of testing time indicated that the cognitive measures were more efficient than the other measures. These results provide at least a preliminary indication that the gap between cognitive measures and conventional measures of clinical performance can be narrowed.

Assessment in Problem-Based Learning Medical Schools: A Literature Review

“…29 PBL leans more toward acquiring integrated knowledge in the context in which it will be used later. 30,31 Therefore, assessment in PBL must challenge traditional assessment systems. 2,15,32,33 It should focus less on the content, recall of isolated facts, and outcomes and focus more on the processes and performances in context, because the fact that students know something is no assurance that they know when or how to use that information.…”

The use of interprofessional peer examiners in an objective structured clinical examination: Can dental students act as examiners?

“…However, in the assessment process the performance of the examinees as well as the examiners changes. 19 Examiners with experience often judge performance in the light of their own experience. 7 There may be concern that the final EDUCATION student examiners same as the median score of marks awarded by the member of staff was not rejected in each of the three pairings A, B, C. Non-parametric 95% confidence intervals for the difference in scores in the three pairings were: A: (-0.5, 0); B: (-0.5, 0.5); C: (-0.5, 0).…”

“…Norman has indicated that breaking down the task into discrete parts using a checklist does not necessarily make the assessment more reliable than a global rating. 19 This may have influenced the results in this pilot. Refinements to the checklist and the use of a global rating scale may help to increase the reliability of this station.…”

When enough is enough: a conceptual basis for fair and defensible practice performance assessment

“…In terms of validity or what the assessment really measures, the format appears to be relatively unimportant 13–16 . If the same content is assessed, it is not how things are asked but what things are asked that decides which competency is being measured.…”