Validity Evidence for a Patient Note Scoring Rubric Based on the New Patient Note Format of the United States Medical Licensing Examination

Abstract

There is initial validity evidence for use of this rubric to score local clinical exams that are based on the new USMLE patient note format.

Characteristics and Implications of Diagnostic Justification Scores Based on the New Patient Note Format of the USMLE Step 2 CS Exam

“…In response to this change, the University of Illinois updated the patient note format used in its graduation competency exam (GCE) to mirror that of the USMLE and developed a rubric to score the new note format. Initial validity evidence for that rubric was reported by Park et al 10 The purpose of the present study was to obtain additional validity evidence for the UIC patient note rubric by investigating the psychometric characteristics and curricular implications of the diagnostic justification scores in the context of the USMLE 2012 format for the postencounter notes.…”

“…10 We found that scoring the new diagnostic justification task of the patient note had benefits for both the student assessment and curriculum evaluation functions of our GCE. The item analysis indicated that of the four tasks in the patient note (Documenting the history and PE, DDx, Justification, and Workup), the justification task was the most predictive of overall GCE performance and provided unique information about students' clinical reasoning abilities.…”

“…A full description of the development of the rubric and initial validity evidence was published by Park et al 10 The previously published version of the rubric was modified on the basis of feedback from faculty raters; the original "DDX" section, which included an integrated rating of both Table 1 Example of a UIC Note Scoring Rubric a Score Distribution for an Abdominal Pain Case, UIC Graduation Competency Exam, May 2014…”

“…a This rubric includes modifications to the previously published rubric10 ; see text for details. b N = 177 third-year students.…”

Studying documentation

“…Success is measured by clinician satisfaction, percent adoption, and note length or timing; note quality is judged using a checklist derived from professional surveys. [5][6][7][8][9][10][11][12][13][14][15] End points like these will often make 1 document look better than another in a "results" section, but it is the relation between communication strategies and healthcare outcomes that determines whether 1 approach or another is of benefit to the patient.…”