A method to quantify and compare clinicians’ assessments of patient understanding during counseling of standardized patients

Abstract

OBJECTIVES-to introduce a method for quantifying clinicians' use of assessment of understanding (AU) questions, and to examine medicine residents' AU usage during counseling of standardized patients about prostate or breast cancer screening.METHODS-Explicit-criteria abstraction was done on 86 transcripts, using a data dictionary for 4 AU types. We also developed a procedure for estimating the "load" of informational content for which the clinician has not yet assessed understanding.RESULTS-Duplicate abstraction revealed reliability κ=0.96. Definite criteria for at least one AU were found in 68/86 transcripts (79%). Of these, 2 transcripts contained a request for a teach-back ("what is your understanding of this?"), 2 contained an open-ended AU, 46 (54%) contained only a close-ended AU, and 18 (21%) only contained an "OK?" question. The load calculation identified long stretches of conversation without an AU.CONCLUSION-Many residents' transcripts lacked AUs, and included AUs were often ineffectively phrased or inefficiently timed. Many patients may not understand clinicians, and many clinicians may be unaware of patients' confusion.PRACTICE IMPLICATIONS-Effective AU usage is important enough to be encouraged by training programs and targeted by population-scale quality improvement programs. This quantitative method should be useful in population-scale measurement of AU usage.

Benefit of Report Card Feedback After Point-of-Care Assessment of Communication Quality Indicators

“…To fill the gap between initial training and long-term practice, we explored the applicability of the Comm QA approach, which uses methods that are straightforward enough to be implemented by existing personnel on a lean budget so that health care organizations can afford to continue them as part of ongoing QI efforts. [16][17][18][19][20][21][22][23][24][25][26][27][28] The results of this pilot project confirmed our earlier findings 27 that a modest Comm QA intervention can produce quantifiable improvements in specific communication behaviors. We observed nearly a doubling of jargon explanations and a 38% relative improvement in the timing of explanations.…”

“…[16][17][18][19][20][21][22][23][24][25][26][27][28] For example, it is feasible to use communication quality indicators with a heterogeneous set of conversational topics. Feedback was limited to only two groups of quality indicators, reducing the chance that participants would be overwhelmed by a large number of details.…”

“…Abstraction process used previously demonstrated explicit-criteria procedures. [16][17][18][19][20][21][22][23][24] Abstractors were blind to whether transcripts were from baseline or follow-up encounters.…”

“…For regulatory reasons the project was considered to have two components: a QI component based on earlier research, [16][17][18][19][20][21][22][23][24][25][26][27][28] and a secondary research analysis that would use the data from the QI effort to develop new measures. The QI component did not require informed consent from physicians, but the secondary research analysis required informed consent from both physicians and patients.…”

“…The result is an approach that the first author has coined "Communication Quality Assurance" (Comm QA). [16][17][18][19][20][21][22][23][24][25][26][27][28] Comm QA methods are designed to be objective, quantitatively reliable, transparent and unobtrusive enough to be acceptable to busy clinicians as well as straightforward enough to be implemented by existing personnel on a lean budget. Communication behaviors are operationalized using an objective "quality indicator" approach instead of subjective rating scales.…”