Background
Physician communication failures during transfers of patients from the intensive care unit (ICU) to the general ward are common and can lead to adverse events. Efforts to improve written handoffs during these transfers are increasingly prominent, but no instruments have been developed to assess the quality of physician ICU–ward transfer notes.
Objective
To collect validity evidence for the modified nine-item Physician Documentation Quality Instrument (mPDQI-9) for assessing ICU–ward transfer note usefulness across several hospitals.
Methods
Twenty-four physician raters independently used the mPDQI-9 to grade 12 notes collected from three academic hospitals.
A priori
, we excluded the “up-to-date” and “accurate” domains, because these could not be assessed without giving the rater access to the complete patient chart. Assessments therefore used the domains “thorough,” “useful,” “organized,” “comprehensible,” “succinct,” “synthesized,” and “consistent.” Raters scored each domain on a Likert scale ranging from 1 (
low
) to 5 (
high
). The total mPDQI-9 was the sum of these domain scores. The primary outcome was the raters’ perceived clinical utility of the notes, and the primary measures of interest were criterion validity (Spearman’s ρ) and interrater reliability (intraclass correlation [ICC]).
Results
Mean mPDQI-9 scores by note ranged from 19 (SD = 5.5) to 30 (SD = 4.2). Mean note ratings did not systematically differ by rater expertise (for interaction,
P
= 0.15). The proportion of raters perceiving each note as independently sufficient for patient care (the primary outcome) ranged from 33% to 100% across the set of notes. We found a moderately positive correlation between mPDQI-9 ratings and raters’ overall assessments of each note’s clinical utility (ρ = 0.48,
P
< 0.001). Interrater reliability was strong; the overall ICC was 0.89 (95% confidence interval [CI], 0.80–0.85), and ICCs were similar among reviewer groups. Finally, Cronbach’s α was 0.87 (95% CI, 0.84–0.89), indicating good internal consistency.
Conclusions
We report moderate validity evidence for the mPDQI-9 to assess the usefulness of ICU–ward transfer notes written by internal medicine residents.