User Experience in Remote Surgical Consultation: Survey Study of User Acceptance and Satisfaction in Real-Time Use of a Telemedicine Service

Abstract

Background Teleguidance, a promising telemedicine service for intraoperative surgical consultation, was planned to scale up at a major academic hospital in partnership with 5 other hospitals. If the service was adopted and used over time, it was expected to provide educational benefits and improve clinical outcomes during endoscopic retrograde cholangiopancreatography (ERCP), which is a technically advanced procedure for biliary and pancreatic disease. However, it is known that seemingly successful innovations can play out differently in new settings, which might cause variability in clinical outcomes. In addition, few telemedicine services survive long enough to deliver system-level outcomes, the causes of which are not well understood. Objective We were interested in factors related to usability and user experience of the telemedicine service, which might affect adoption. Therefore, we investigated perceptions and responses to the use and anticipated use of a system. Technology acceptance, a construct referring to how users perceive a technology’s usefulness, is commonly considered to indicate whether a new technology will actually be used in a real-life setting. Satisfaction measures were used to investigate whether user expectations and needs have been met through the use of technology. In this study, we asked surgeons to rate the perceived usefulness of teleguidance, and their satisfaction with the telemedicine service in direct conjunction with real-time use during clinical procedures. Methods We designed domain-specific measures for perceived usefulness and satisfaction, based on performance and outcome measures for the clinical procedure. Surgeons were asked to rate their user experience with the telemedicine service in direct conjunction with real-time use during clinical procedures. Results In total, 142 remote intraoperative consultations were conducted during ERCP procedures at 5 hospitals. The demand for teleguidance was more pronounced in cases with higher complexity. Operating surgeons rated teleguidance to have contributed to performance and outcomes to a moderate or large extent in 111 of 140 (79.3%) cases. Specific examples were that teleguidance was rated as having contributed to intervention success and avoiding a repeated ERCP in 23 cases, avoiding 3 PTC, and 11 referrals, and in 11 cases, combinations of these outcomes. Preprocedure beliefs about the usefulness of teleguidance were generally lower than postprocedure satisfaction ratings. The usefulness of teleguidance was mainly experienced through practical advice from the consulting specialist (119/140, 85%) and support with assessment and decision-making (122/140, 87%). Conclusions Users’ satisfaction with teleguidance surpassed their initial expectations, mainly through contribution to nontechnical aspects of performance, and through help with general assessment. Teleguidance shows the potential to improve performance and outcomes during ERCP. However, it takes hands-on experience for practitioners to understand how the new telemedicine service contributes to performance and outcomes.

Should Expert Surgeon Guidance Be Given Remotely? Comment on “User Experience in Remote Surgical Consultation: Survey Study of User Acceptance and Satisfaction in Real-Time Use of a Telemedicine Service”

“…Even though some of these issues were resolved either through connection restarting or calling medical technicians, these unnecessary delays increase patients’ time under anesthesia, which is known to increase perioperative complications such as infection [ 3 ]. Indeed, this could explain the 4 (2.8%) cases that had postoperative complications; although, we cannot confirm this as a causal inference [ 1 ]. Furthermore, due to the limitations in the content being transferred at any one time via the service and the lack of 3D visualization, we caution against the use of remote expert guidance in particularly difficult surgical decision-making scenarios.…”

“…We recently read with great interest the article “User Experience in Remote Surgical Consultation: Survey Study of User Acceptance and Satisfaction in Real-Time Use of a Telemedicine Service” by Aminoff et al [ 1 ], where a real-time telemedicine service was used for intraoperative surgical consultation assistance during endoscopic retrograde cholangiopancreatography procedures. The authors investigated the surgeons’ preprocedure expectations of how the service would aid operative performance and patient outcomes, and their satisfaction after use of the service.…”

“…Although this study presents interim results, we are concerned that the perceived future use of the service may be overestimated. Expert surgeons had to book out their time in advance to participate, and when the sample was asked about the perceived demand for teleguidance, 80% said they believed there would be no demand for it or were unsure of the demand [ 1 ]. This coupled with the fact that sites were encouraged to use the service due to being part of a trial makes it unclear as to whether the telemedicine service would be used in practice.…”

“…Although novice endoscopic retrograde cholangiopancreatographists demonstrated an increase in their level of expertise during the study period, we argue that this intervention may be time-inefficient, with 81.1% of the procedure time being spent on receiving guidance [ 1 ]. Furthermore, no follow-up data is available to predict whether skills would be maintained or regress if tele-guidance were not implemented, which casts doubt on the service’s training value.…”