State-level COVID-19 vaccination rates among older adults have been uneven in the United States. Due to the immunocompromised nature of older adults, vaccine hesitancy increases the risk of morbidity and mortality. This study aims to determine the association between the social determinants of health, the structural determinants of health, and COVID-19 vaccine hesitancy among older adults in the United States. Secondary data from the Health and Retirement Study (HRS) dataset were used. A descriptive analysis and multinomial multivariable logistic regression were performed to examine the association of the independent variables—gender, age, race, immigration status, marital status, broadband internet access, social security income, Medicare coverage, education, and frequency of religious service—with the dependent variable, vaccine hesitancy. Compared to the respondents with no vaccine hesitancy and without the specific predictor, the respondents who reported religious attendance at least once/week were more likely to be “somewhat hesitant”, divorced respondents had higher odds of being “somewhat hesitant”, and older adults aged 65–74 years were more likely to be “very hesitant” or “somewhat hesitant” about the COVID-19 vaccine. Compared to the respondents with no vaccine hesitancy and without the specific predictor, females had higher odds of being “very hesitant”, “somewhat hesitant”, or a “little hesitant”, and African Americans were more likely to be “very hesitant”, “somewhat hesitant”, or a “little hesitant” about the COVID-19 vaccine. Addressing these factors may limit the barriers to vaccine uptake reported among older adults and improve herd immunity among the immunocompromised population.