Billions of doses of COVID-19 vaccines have been administered around the world, dramatically reducing SARS-CoV-2 incidence in some settings. Many studies suggest vaccines provide a high degree of protection against infection and disease, but precise estimates vary and studies differ in design, outcomes measured, dosing regime, location, and circulating virus strains. Here we conduct a systematic review of COVID-19 vaccines as of August 2021. We included efficacy data from Phase 3 clinical trials for 13 vaccines within the WHO Emergency Use Listing evaluation process and real-world effectiveness for 5 vaccines with observational studies meeting inclusion criteria. Vaccine metrics collected include effects against asymptomatic infection, any infection, symptomatic COVID-19, and severe outcomes including hospitalization and death, for both partial and complete vaccination, and against SARS-CoV-2 variants of concern. In addition, we review the epidemiological principles behind the design and interpretation of vaccine effects, and explain important sources of heterogeneity between studies.