Vaccine hesitancy is one of the top ten greatest threats to global health. During the COVID-19 era, vaccine hesitancy poses substantial risks, especially in visible minorities, who are disproportionately affected by the pandemic. Although evidence of vaccine hesitancy exists, there is minimal focus on visible minorities and the reasons for hesitancy in this group are unclear. Identifying these populations and their reasons for vaccine hesitancy is crucial in improving vaccine uptake and curbing the spread of COVID-19. This scoping review follows a modified version of the Arksey and O’Malley strategy. Using comprehensive search strategies, advanced searches were conducted on Medline, CINAHL, and PubMed databases to acquire relevant articles. Full-text reviews using inclusion and exclusion criteria were performed to extract themes of vaccine hesitancy. Themes were grouped into factors using thematic qualitative analysis and were objectively confirmed by principal component analysis (PCA). To complement both analyses, a word cloud of titles and abstracts for the final articles was generated. This study included 71 articles. Themes were grouped into 8 factors and the top 3 recurring factors were safety and effectiveness of the vaccine, mistrust, and socioeconomic characteristics. Shedding light on these factors could help mitigate health inequities and increase overall vaccine uptake worldwide through interventions and policies targeted at these factors. Ultimately, this would help achieve global herd immunity.