Evaluating Medical Devices Remotely: Current Methods and Potential Innovations

Abstract

Objective We present examples of laboratory and remote studies, with a focus on studies appropriate for medical device design and evaluation. From this review and description of extant options for remote testing, we provide methods and tools to achieve research goals remotely. Background The FDA mandates human factors evaluation of medical devices. Studies show similarities and differences in results collected in laboratories compared to data collected remotely in non-laboratory settings. Remote studies show promise, though many of these are behavioral studies related to cognitive or experimental psychology. Remote usability studies are rare but increasing, as technologies allow for synchronous and asynchronous data collection. Method We reviewed methods of remote evaluation of medical devices, from testing labels and instruction to usability testing and simulated use. Each method was coded for the attributes (e.g., supported media) that need consideration in usability studies. Results We present examples of how published usability studies of medical devices could be moved to remote data collection. We also present novel systems for creating such tests, such as the use of 3D printed or virtual prototypes. Finally, we advise on targeted participant recruitment. Conclusion Remote testing will bring opportunities and challenges to the field of medical device testing. Current methods are adequate for most purposes, excepting the validation of Class III devices. Application The tools we provide enable the remote evaluation of medical devices. Evaluations have specific research goals, and our framework of attributes helps to select or combine tools for valid testing of medical devices.

Going Remote—Demonstration and Evaluation of Remote Technology Delivery and Usability Assessment With Older Adults: Survey Study

“…A significant challenge overcome by our study team was the lack of existing methodologies and guidelines available for remote usability testing with older adults. Recently, McLaughlin et al [ 8 ] outlined various tools available for different kinds of remote usability testing of medical devices and recommended that study personnel be available for phone support during tests with older adults. Beyond this advice, there is no clear guidance from human-computer interaction, usability, mHealth, or other literature on how to operationalize a fully remote study protocol for older adult participants.…”

“…The pandemic also disrupted research, forcing projects to suspend operations or implement remote noncontact methods [ 5 - 7 ]. Paradoxically, some studies of mobile technology faced difficulties converting to entirely remote technology delivery and usability assessment, revealing opportunities for technology projects to experiment with remote operations [ 8 ].…”

“…Potentially useful existing methods to access and evaluate technology include online questionnaires, online forums, self-kept diaries, email, telephone calls, and postal delivery [ 11 , 15 - 17 ]. More recently, videoconferencing tools are used to replicate in-person, moderated usability testing [ 8 , 11 , 17 ].…”

Going Remote—Demonstration and Evaluation of Remote Technology Delivery and Usability Assessment With Older Adults: Survey Study (Preprint)

“…The pandemic also disrupted research, forcing projects to suspend operations or implement remote noncontact methods [5][6][7]. Paradoxically, some studies of mobile technology faced difficulties converting to entirely remote technology delivery and usability assessment, revealing opportunities for technology projects to experiment with remote operations [8].…”

“…Potentially useful existing methods to access and evaluate technology include online questionnaires, online forums, self-kept diaries, email, telephone calls, and postal delivery [11,[15][16][17]. More recently, videoconferencing tools are used to replicate in-person, moderated usability testing [8,11,17].…”