Background Adoption and evaluation of contact tracing tools based on information and communications technology may expand the reach and efficacy of traditional contact tracing methods in fighting COVID-19. The Dutch Ministry of Health, Welfare and Sports initiated and developed CoronaMelder, a COVID-19 contact tracing app. This app is based on a Google/Apple Exposure Notification approach and aims to combat the spread of the coronavirus among individuals by notifying those who are at increased risk of infection due to proximity to someone who later tests positive for COVID-19. The app should support traditional contact tracing by faster tracing and greater reach compared to regular contact tracing procedures. Objective The main goal of this study is to investigate whether the CoronaMelder is able to support traditional contact tracing employed by public health authorities. To achieve this, usability tests were conducted to answer the following question: is the CoronaMelder user-friendly, understandable, reliable and credible, and inclusive? Methods Participants (N=44) of different backgrounds were recruited: youth with varying educational levels, youth with an intellectual disability, migrants, adults (aged 40-64 years), and older adults (aged >65 years) via convenience sampling in the region of Twente in the Netherlands. The app was evaluated with scenario-based, think-aloud usability tests and additional interviews. Findings were recorded via voice recordings, observation notes, and the Dutch User Experience Questionnaire, and some participants wore eye trackers to measure gaze behavior. Results Our results showed that the app is easy to use, although problems occurred with understandability and accessibility. Older adults and youth with a lower education level did not understand why or under what circumstances they would receive notifications, why they must share their key (ie, their assigned identifier), and what happens after sharing. In particular, youth in the lower-education category did not trust or understand Bluetooth signals, or comprehend timing and follow-up activities after a risk exposure notification. Older adults had difficulties multitasking (speaking with a public health worker and simultaneously sharing the key in the app). Public health authorities appeared to be unprepared to receive support from the app during traditional contact tracing because their telephone conversation protocol lacks guidance, explanation, and empathy. Conclusions The study indicated that the CoronaMelder app is easy to use, but participants experienced misunderstandings about its functioning. The perceived lack of clarity led to misconceptions about the app, mostly regarding its usefulness and privacy-preserving mechanisms. Tailored and targeted communication through, for example, public campaigns or social media, is necessary to provide correct information about the app to residents in the Netherlands. Additionally, the app should be presented as part of the national coronavirus measures instead of as a stand-alone app offered to the public. Public health workers should be trained to effectively and empathetically instruct users on how to use the CoronaMelder app.

A Digital electronic Coach (eCoach) app was built and evaluated during the Covid-19 pandemic in The Netherlands. Its aim was to provide support for AQ1 individuals that had to either quarantine or self-isolate after a positive corona test or an indication of a heightened risk of infection. The coach ("IsolationCoach"), AQ2 its value and uses were evaluated in 29 semi-structured interviews with individuals who had quarantined or isolated themselves or were part of the general Dutch public. Three main findings emerge. First, participants found value in a digital coach that would help them comply with quarantine or isolation instructions and provided information on the practical challenges of organizing their quarantine or isolation. Second, the usage of the app, which gradually and conditionally provides relevant information as opposed to conventional paper pamphlets/email, was greatly appreciated. Third, participants experienced a need for mental support during their period of isolation or quarantine, and this could at least partially be filled by the eCoach, which provided emotional support through a Socratic method styled form of self-reflection. It was beneficial that the app was implemented rapidly within weeks using a ready-to-use platform and that its content was assessed by experts from various health-related disciplines prior to rollout. Yet, for large-scale implementation, an integrated vision and digital strategy is needed to align forms of support by the health authorities.

The coronavirus disease (COVID-19) has led to a global pandemic. AQ1 Many countries are using contact tracing as one of the interventions to control virus transmission. In conventional contact tracing, index cases with positive test results are asked to provide contact information of close contacts who were at risk of acquiring infection from the index case, within a given time period before the AQ2 positive test result. Mobile contact tracing apps can augment traditional contact tracing, as they exchange contact information wirelessly and semi-automatically, and allow for quicker warning of at-risk contacts. AQ3 However, little is known about user acceptance and perception of such contact tracing apps, or how people perceive the advice presented therein. In July 2020, the beta version of the official Dutch corona notification app was usability tested with participants from various age groups. This paper presents the results of the objective eye tracking measurements executed during these usability tests. The results show that both health and process related information is easily overlooked in the app by participants. A closed-loop contact with the builders of the app allowed for a direct feedback channel and quick improvements during the development stage.

BACKGROUND Adoption and evaluation of ICT-based contact tracing tools may expand the reach and efficacy of traditional contact tracing methods in fighting COVID-19. The Dutch Ministry of Health, Welfare and Sports (HWS) initiated and developed a COVID-19 contact tracing app: CoronaMelder. This app is based on Google/Apple exposure notification approach and aims to combat the spread of the COVID-19 virus among citizens, by notifying citizens who were at increased risk of infection because they were close by someone who was later tested positive for COVID-19. The app should support the traditional contact tracing by quicker tracing and reaching more people than regular contact tracing procedures. OBJECTIVE The main goal of this study is to investigate whether the CoronaMelder is able to support traditional contact tracing of Public Health Authorities (PHAs). To achieve this, usability tests were conducted aimed at answering the following question: Is the CoronaMelder user-friendly, understandable, reliable and credible, and inclusive? METHODS Participants (n=44) with different backgrounds were recruited: young people with a lower or higher level of education, young people with an intellectual disability, migrants, adults (40-64 years) and elderly (65> years) via convenience sampling in the CoronaMelder test region Twente, The Netherlands. The app was evaluated with scenario-based think-aloud usability tests with additional interviews. Findings were recorded via voice recordings, observation notes, the Dutch User Experience Questionnaire (UEQ-Dutch) and some participants wore eye trackers to measure gaze behavior. RESULTS Our results show that the app is easy to use. Yet, problems occurred with understandability and accessibility. Elderly and young people with a lower level of education do not understand why or when they receive notifications, or why they must share the key, and what happens after sharing. Especially young people with a lower level of education did not trust and understand the Bluetooth signals, timing and follow-up activities after risk exposure notification and elderly had difficulties in multitasking (contact with PHAs simultaneously with sharing key in app). PHAs appeared unprepared to be supported by the app in traditional contact tracing, because their telephone conversation protocol lacks guidance, explanation, and empathy. CONCLUSIONS The study indicated that the app is easy to use, but participants have misconceptions about its functioning. The perceived lack of clarity led to misconceptions of the app, mostly regarding its usefulness or privacy-preserving mechanisms. Tailored and target group specified communication, in forms of public campaigns or social media, is necessary to provide correct information about the app to Dutch citizens. Additionally, the app should be presented as part of the package of national corona measures, instead of just as a stand-alone app provided to the public. To succeed, PHA workers should be trained to effectively and empathically instruct users to warn others by using the CoronaMelder app.

Background The Dutch CoronaMelder (CM) app is the official Dutch contact-tracing app (CTA). It has been used to contain the spread of the SARS-CoV-2 in the Netherlands. It allows its users and those of connected apps to anonymously exchange warnings about potentially high-risk contacts with individuals infected with the SARS-CoV-2. Objective The goal of this mixed methods study is to understand the use of CTA in the pandemic and its integration into the Municipal Health Services (MHS) efforts of containment through contact tracing. Moreover, the study aims to investigate both the motivations and user experience–related factors concerning adherence to quarantine and isolation measures. Methods A topic analysis of 56 emails and a web-based survey of 1937 adults from the Netherlands, combined with a series of 48 in-depth interviews with end users of the app and 14 employees of the Dutch MHS involved in contact tracing, were conducted. Mirroring sessions were held (n=2) with representatives from the development (n=2) and communication teams (n=2) responsible for the creation and implementation of the CM app. Results Topic analysis and interviews identified procedural and technical issues in the use of the CTA. Procedural issues included the lack of training of MHS employees in the use of CTAs. Technical issues identified for the end users included the inability to send notifications without phone contact with the MHS, unwarranted notifications, and nightly notifications. Together, these issues undermined confidence in and satisfaction with the app’s use. The interviews offered a deeper understanding of the various factors at play and their effects on users; for example, the mixed experiences of the app’s users, the end user’s own fears, and uncertainties concerning the SARS-CoV-2; problematic infrastructure at the time of the app’s implementation on the side of the health services; the effects of the society-wide efforts in containment of the SARS-CoV-2 on the CM app’s perception, resulting in further doubts concerning the app’s effectiveness among MHS workers and citizens; and problems with adherence to behavioral measures propagated by the app because of the lack of confidence in the app and uncertainty concerning the execution of the behavioral measures. All findings were evaluated with the app’s creators and have since contributed to improvements. Conclusions Although most participants perceived the app positively, procedural and technical issues identified in this study limited satisfaction and confidence in the CM app and affected its adoption and long-term use. Moreover, these same issues negatively affected the CM app’s effectiveness in improving compliance with behavioral measures aimed at reducing the spread of the SARS-CoV-2. This study offers lessons learned for future eHealth interventions in pandemics. Lessons that can aid in more effective design, implementation, and communication for more effective and readily adoptable eHealth applications.