The worldwide COVID-19 vaccination program is already underway, raising hopes and aspirations to contain the spread of the COVID-19 pandemic that halted economic and social activities. However, the issue of vaccine effectiveness and its side-effects is influencing the potential acceptance of vaccines. In this uncertain situation, we used data from a nationwide survey in Japan during February 2021, following the Japanese government’s initial phase of COVID-19 vaccination. Our results show that 47% of the respondents are willing to take a vaccine once it is available, while 22% are not willing and another 31% remain indecisive. Our ordered probit regression results show that demographic, socioeconomic, and behavioral variables such as gender, age, subjective health status, children, household income, household assets, financial literacy, future anxiety, and myopic view of the future are associated with willingness to take a COVID-19 vaccine. Our findings suggest that Japan’s government should not adopt a one-size-fits-all policy to promote the vaccination program, but rather target people with specific socioeconomic backgrounds who are less willing and more hesitant to take a vaccine.