Contact tracing is a non-pharmaceutical intervention (NPI) widely used in the control of the COVID-19 pandemic. Its effectiveness may depend on a number of factors including the proportion of contacts traced, delays in tracing, the mode of contact tracing (e.g. forward, backward or bidirectional contact training), the types of contacts who are traced (e.g. contacts of index cases or contacts of contacts of index cases), or the setting where contacts are traced (e.g. the household or the workplace). We performed a systematic review of the evidence regarding the comparative effectiveness of contact tracing interventions. 78 studies were included in the review, 12 observational (ten ecological studies, one retrospective cohort study and one pre-post study with two patient cohorts) and 66 mathematical modelling studies. Based on the results from six of the 12 observational studies, contact tracing can be effective at controlling COVID-19. Two high quality ecological studies showed the incremental effectiveness of adding digital contact tracing to manual contact tracing. One ecological study of intermediate quality showed that increases in contact tracing were associated with a drop in COVID-19 mortality, and a pre-post study of acceptable quality showed that prompt contact tracing of contacts of COVID-19 case clusters / symptomatic individuals led to a reduction in the reproduction number R. Within the seven observational studies exploring the effectiveness of contact tracing in the context of the implementation of other non-pharmaceutical interventions, contact tracing was found to have an effect on COVID-19 epidemic control in two studies and not in the remaining five studies. However, a limitation in many of these studies is the lack of description of the extent of implementation of contact tracing interventions. Based on the results from the mathematical modelling studies, we identified the following highly effective policies: (1) manual contact tracing with high tracing coverage and either medium-term immunity, highly efficacious isolation/quarantine and/ or physical distancing (2) hybrid manual and digital contact tracing with high app adoption with highly effective isolation/ quarantine and social distancing, (3) secondary contact tracing, (4) eliminating contact tracing delays, (5) bidirectional contact tracing, (6) contact tracing with high coverage in reopening educational institutions. We also highlighted the role of social distancing to enhance the effectiveness of some of these interventions in the context of 2020 lockdown reopening. While limited, the evidence from observational studies shows a role for manual and digital contact tracing in controlling the COVID-19 epidemic. More empirical studies accounting for the extent of contact tracing implementation are required.