Effectiveness evaluation of digital contact tracing for COVID-19 in New South Wales, Australia

Abstract

Background Digital proximity tracing apps were rolled out early in the COVID-19 pandemic in many countries to complement conventional contact tracing. Empirical evidence about their benefits for pandemic response remains scarce. We evaluated the effectiveness and usefulness of COVIDSafe, Australia's national smartphone-based proximity tracing app for COVID-19. Methods In this prospective study, done in New South Wales (NSW), Australia, we included all individuals in the state who were older than 12 years with confirmed, locally acquired SARS-CoV-2 infection between May 4 and Nov 4, 2020. We used data from the NSW Notifiable Conditions Information Management System, the national COVIDSafe database, and information from case interviews, including information on app usage, the number of app-suggested contacts, and the number of app-suggested contacts determined by public health staff to be actual close contacts. We calculated the positive predictive value and sensitivity of COVIDSafe, its additional contact yield, and the number of averted public exposure events. Semi-structured interviews with public health staff were done to assess the app's perceived usefulness. Findings There were 619 confirmed COVID-19 cases with more than 25 300 close contacts identified by conventional contact tracing during the study period. COVIDSafe was used by 137 (22%) cases and detected 205 contacts, 79 (39%) of whom met the close contact definition. Its positive predictive value was therefore 39%. 35 (15%) of the 236 close contacts who could have been expected to have been using the app during the study period were identified by the app, making its estimated sensitivity 15%. 79 (0·3%) of the estimated 25 300 contacts in NSW were app-suggested and met the close contact definition. The app detected 17 (<0·1%) additional close contacts who were not identified by conventional contact tracing. COVIDSafe generated a substantial additional perceived workload for public health staff and was not considered useful. Interpretation The low uptake of the app among cases probably led to a reduced sensitivity estimate in our study, given that only contacts who were using the app could be detected. COVIDSafe was not sufficiently effective to make a meaningful contribution to the COVID-19 response in Australia's most populous state over a 6 month period. We provide an empirical evaluation of this digital contact tracing app that questions the potential benefits of digital contact tracing apps to the public health response to COVID-19. Effectiveness evaluations should be integrated into future implementations of proximity contact tracing systems to justify their investment. Funding New South Wales Ministry of Health (Australia); National Health and Medical Research Council (Australia)

Learning from the initial deployment of digital contact tracing apps

“…Therefore, conventional contact tracing was extraordinarily effective, with more than 40 close contacts identified per COVID-19 case on average. 4 This is in sharp contrast with the fewer than two close contacts per case identified by conventional contact tracing in the UK study. 5 The contact tracing authorities in NSW were never overwhelmed during the study period and, therefore, the relative interest and the cost–benefit balance of digital contact tracing were probably less apparent than in other countries.…”

“…In The Lancet Public Health , Florian Vogt and colleagues 4 report the findings from a prospective study of the deployment of COVIDSafe, Australia's national COVID-19 tracing app, in the state of New South Wales (NSW). The app was based on a centralised approach, in which the information of contacts is uploaded to a common database and accessed by public health authorities for risk assessment and notification.…”

“…9 Its performance depends on the functioning of the technology, but also on adoption by the users and how it integrates into the public health response. Vogt and colleagues 4 discuss in detail the issues that might have limited the impact of COVIDSafe, thus providing precious lessons learnt. The authors acknowledge that if a better real-world piloting of the app had been done before its launch, this might have helped to improve app performances.…”

The effect of notification window length on the epidemiological impact of COVID-19 contact tracing mobile applications

“…Previous studies have demonstrated that the extent of active usage of mobile contact tracing apps has a large impact on their effectiveness 18 , 19 . The time to notification of exposed contacts also plays an integral role for SARS-CoV-2 20 ; this is corroborated by our results regarding the contrasting reduction in R * based on whether a user inputs their positive result from an LFT or from a PCR test (where the PCR test involves a delay between the test date and the positive test result).…”

COVID-19 contact-tracing smartphone application usage—The New Zealand COVID Tracer experience

“…Digital contact tracing can augment an existing contact tracing system ( Plank et al, 2020 ) but this presupposes good integration with the existing tracing infrastructure ( Lee & Kim, 2021 ) and rapid response once a contact has been identified. There are circumstances in which digital tracing has not proved helpful ( Vogt et al, 2022 ) however.…”