Background: By 11 March 2022, there were 450,229,635 Coronavirus disease (COVID-19) cases and 6,019,085 deaths globally, with Nigeria reporting 254,637 cases and 3142 deaths. One of the essential healthcare services that has been impacted by the pandemic is childhood routine immunization. According to the 2018 National Demographic and Health Survey, only 31% of children aged 12–23 months were fully vaccinated in Nigeria, and 19% of eligible children haven’t received any vaccination in the country. A further decline in coverage due to the pandemic can significantly increase the risk of vaccine-preventable-disease outbreaks among children in Nigeria. To mitigate such an occurrence, it is imperative to urgently identify how the pandemic and the response strategies have affected vaccination services, hence, the goal of the study. Methods: The research method is qualitative, including in-depth interviews of healthcare workers and focus group discussions (FGD) with caregivers of children 0–23 months. We selected one state from each of the three zones of Nigeria: northern, central, and southern. Within each state, 10 local government areas and 20 healthcare facilities were purposively selected. In each facility, 10 healthcare workers were invited for interview. Overall, 517 healthcare workers were interviewed. For the focus group discussion, 30 communities were selected. Within each selected community, six consenting caregivers were included. Overall, 180 caregivers participated. The data were analyzed using thematic inductive content analysis. Results: Three significant impacts that were observed are: difficulties of accessibility to immunization services, declining immunization demand and uptake among caregivers due to varying factors, and erosion of vaccine confidence among both caregivers and healthcare workers. Movement restriction and lockdown had major, numerous impacts, such as decreased general healthcare services delivery, increased transportation costs, fewer engagements that promotes vaccine uptake, and cessation of mobile vaccination campaigns that targeted hard-to-reach communities. Moreover, misinformation, conspiracy beliefs about the pandemic and COVID-19 vaccines, and risk perception negatively influenced general vaccine confidence. Conclusion: The results from this early impact study show immunization was directly affected by the pandemic and provided insights into areas where interventions are needed for recovery.