Background COVID-19 restrictions severely curtailed empirical endeavors that involved in-person interaction, such as usability testing sessions for technology development. Researchers and developers found themselves using web-based moderation for usability testing. Skilled remote moderators and observers are fundamental in this approach. However, to date, more empirical work is needed that captures the perceptions and support needs of moderators and observers in testing situations. Objective The aim of this paper was to identify remote moderator and observer participant experiences and their use of certain tools to capture feedback of users as they interact with the web browser application. Methods This research is part of a broader study on an educational web browser application for nursing students to learn perspective taking and enhance their perceptual understanding of a dialogue partner’s thoughts and feelings. The broader study used a quantitative and think-aloud qualitative problem-discovery usability study design. This case study explored written accounts of the remote moderator and observer participants regarding their roles, experiences, and reactions to the testing protocol and their suggestions for improved techniques and strategies for conducting remote usability testing. Content analysis was used to analyze participants’ experiences in the usability testing sessions. Results We collected data from 1 remote moderator and 2 remote observers. Five themes were identified: dealing with personal stressors, dealing with user anxiety, maintaining social presence, ethical response to the study protocol, and communication during sessions. The participants offered recommendations for the design of future remote testing activities as well as evidence-informed training materials for usability project personnel. Conclusions This study’s findings contribute to a growing body of endeavors to understand human-computer interaction and its impact on remote moderator and observer roles. As technology rapidly advances, more remote usability testing will occur where the knowledge gleaned in this study can have an impact. Recommendations based on moderator and observer participant perspectives identify the need for more evidence-informed training materials for their roles that focus on web-based interpersonal communication skills, execution of user testing protocols, troubleshooting technology and test user issues, proficiency in web conferencing platforms, behavior analysis and feedback technologies, and time management.

Background The prepandemic period saw a rise in web-based teaching. However, web-based tools for teaching the essential clinical skill of cognitive empathy (also known as perspective taking) remain limited. More of these tools are needed and require testing for ease of use and understanding by students. Objective This study aimed to evaluate the usability of the In Your Shoes web-based empathy training portal application for students using quantitative and qualitative methods. Methods This 3-phase formative usability study used a mixed methods design. In mid-2021, we conducted a remote observation of student participants interacting with our portal application. Their qualitative reflections were captured, followed by data analysis and iterative design refinements of the application. Overall, 8 third- and fourth-year nursing students from an undergraduate baccalaureate program at a Canadian university, in the western province of Manitoba, were included in this study. Participants in phases 1 and 2 were remotely observed by 3 research personnel while engaged in predefined tasks. In phase 3, two student participants were asked to use the application as they liked in their own environments, after which a video-recorded exit interview with a think-aloud process was conducted as participants responded to the System Usability Scale. We calculated descriptive statistics and performed content analysis to analyze the results. Results This small study included 8 students with a range of technology skills. Usability themes were based on participants’ comments on the application’s appearance, content, navigation, and functionality. The biggest issues that participants experienced were with navigating the application’s “tagging” features during video analysis and the length of educational material. We also observed variations in 2 participants’ system usability scores in phase 3. This may be because of their different comfort levels with technology; however, additional research is required. We made iterative refinements to our prototype application (eg, added pop-up messages and provided a narrated video on the application’s “tagging” function) based on participant feedback. Conclusions With increasing engagement in web-based teaching, technology has become an essential medium for receiving health care education. We developed a novel prototype application as a supplemental classroom tool to foster students’ self-directed learning of empathy. This study provided direction for refinements to optimize the usability of and satisfaction with this innovative application. Qualitative feedback revealed favorable input toward learning perspective taking place on the web and helpful recommendations for improving user experiences with the application. We could not fully assess the application’s key functions owing to the COVID-19 protocols. Thus, our next step is to obtain feedback from a larger sample of student users, whose experiences performing “live” video capture, annotation, and analysis will be more authentic and wholesome with the refined application. We discuss our findings in relation to research on nursing education, perspective taking, and adaptive e-learning.

No abstract

BACKGROUND Pre-pandemic saw the rise of online teaching. Yet, online tools for teaching the essential nursing skill of empathy remain limited. Not only are more of these tools needed, but they need to be tested for ease of use and understanding by students. OBJECTIVE To evaluate student usability of the In Your Shoes web-based empathy training portal application with quantitative and qualitative methods. METHODS A mixed methods design was employed in this 3-phase formative usability study. We conducted remote observation of student participants interacting with the training portal application and captured their qualitative reflections followed by data analysis and iterative design refinements to the application. Eight third- and fourth-year nursing students were from an undergraduate baccalaureate program at one mid-Canada university. Student participants in Phases 1 and 2 were remotely observed by research personnel while engaged in pre-defined tasks. In Phase 3, student participants were asked to use the application as they liked in their own environments; this was followed by a video-recorded exit interview with a think aloud process as student participants responded to the System Usability Scale. RESULTS We found ‘above average’ perceived usability with the application. Student participants had a range of technology skills and tended to give one-off comments on application appearance, content, navigation, and functionality. Main challenges included navigation during video-analysis and the length of educational materials. Application refinements were made (e.g., pop-up messages, narrated video) based on feedback. CONCLUSIONS With the rising engagement in online teaching, technology is an essential medium for learning in health care education. We developed a novel tool as a supplemental classroom tool to foster students’ self-directed learning about empathy. This study provided direction for ongoing refinements to enhance the usability and user satisfaction with the application. Our next step is to obtain feedback from users who will experience more authentic experiences doing ‘live’ video-capture, -annotation, and -analysis as these could not be fully evaluated due to COVID protocol. CLINICALTRIAL not applicable. INTERNATIONAL REGISTERED REPORT RR2-10.2196/35319