Real-time Feedback on Nonverbal Clinical Communication

Abstract

SummaryIntroduction: This article is part of the Focus Theme of Methods of Information in Medicine on "Pervasive Intelligent Technologies for Health". Background: Effective nonverbal communication between patients and clinicians fosters both the delivery of empathic patient-centered care and positive patient outcomes. Although nonverbal skill training is a recognized need, few efforts to enhance patient-clinician communication provide visual feedback on nonverbal aspects of the clinical encounter. Objectives: We describe a novel approach that uses social signal processing technology (SSP) to capture nonverbal cues in real time and to display ambient visual feedback on control and affiliation -two primary, yet distinct dimensions of interpersonal nonverbal communication. To examine the design and clinician acceptance of ambient visual feedback on nonverbal communication, we 1) formulated a model of relational communication to ground SSP and 2) conducted a formative user study using mixed methods to explore the design of visual feedback. Methods: Based on a model of relational communication, we reviewed interpersonal communication research to map nonverbal cues to signals of affiliation and control evidenced in patient-clinician interaction. Corresponding with our formulation of this theoretical framework, we designed ambient real-time visualizations that reflect variations of affiliation and control. To explore clinicians' acceptance of this visual feedback, we conducted a lab study using the Wizard-of-Oz technique to simulate system use with 16 healthcare professionals. We followed up with seven of those participants through interviews to iterate on the design with a revised visualization that addressed emergent design considerations. Results: Ambient visual feedback on nonverbal communication provides a theoretically grounded and acceptable way to provide clinicians with awareness of their nonverbal communication style. We provide implications for the design of such visual feedback that encourages empathic patient-centered communication and include considerations of metaphor, color, size, position, and timing of feedback. Conclusions: Ambient visual feedback from SSP holds promise as an acceptable means for facilitating empathic patient-centered nonverbal communication.

Improving Medical Students’ Awareness of Their Non-Verbal Communication through Automated Non-Verbal Behavior Feedback

“…However, in the past three decades, non-verbal communication, which accounts for approximately 80% of essential communication between individuals (Gorawara-Bhat et al, 2007), has been attracting increasing attention from the medical education community. Specific non-verbal behaviors, including facial expression and body movement, have been proven to be related to patient satisfaction (DiMatteo et al, 1980) and health outcomes (Hartzler et al, 2014). Within medical education, the performance of certain non-verbal behaviors by medical students, such as "maintaining adequate facial expressions, using affirmative gestures and limiting both unpurposive movements and hand gestures, had a significant positive effect on perception of interview quality during (an) Objective Structure Clinical Examination" (Collins et al, 2011).…”

“…Non-verbal behavior includes every communicative human act other than speech and it can be generally classified into four categories: (1) kinesics: head and body movements, such as facial expressions and gestures, (2) vocalics: non-linguistic vocal cues, such as volume and sound pitch, (3) haptics: body contact, such as handshakes, (4) proxemics: spatial cues, such as doctor-patient distance and body orientation (Hartzler et al, 2014;Burgoon et al, 2016). At this time, there is no consensus on which category of non-verbal behavior is most important in clinical consultations.…”

“…Kinesics behavior includes the movements of the face, head, trunk, hands, and limbs (Hartzler et al, 2014). Face and head movements are among the most salient non-verbal behaviors in patient-doctor consultations.…”

“…Touch, such as a pat on the patient's shoulder, is a communication gesture used by clinicians to express support and sympathy (Hartzler et al, 2014). In tele-conferencing, physical contact is not possible between clinicians and patients.…”

“…In general, close conversational distance indicates an intimate consultation (Hartzler et al, 2014). Some studies shown that patients' satisfaction was greater if the doctor lean toward them (Larsen and Smith, 1981) or sit in a closer distance (Weinberger et al, 1981) in clinical consultations.…”