Background: Having no choice in becoming a caregiver has been associated with higher levels of emotional stress and physical strain. This secondary analysis examined associations between caregivers’ perceived choice and health outcomes. Methods: This study utilized data from caregivers who answered a question about whether they felt they had a choice to take on caring for a care recipient in the Caregiving in the U.S. in 2020 survey. Variables of caregivers’ and recipients’ characteristics, caregiving activities, and health outcomes were extracted. Data were analyzed using descriptive statistics, t-tests, Chi-squared tests, and regression models. Results: More than half of the 1,642 caregivers (54.4%) perceived no choice in becoming a caregiver. Having no choice was associated with higher levels of physical strain and emotional stress, and greater negative impact on caregiver’s health. Other predictive variables of higher physical strain included being a primary caregiver, recipients having more comorbidities, and a higher level of care intensity. Higher education level, household income, number of recipient’s conditions, level of care intensity, and being a primary caregiver were associated with higher levels of emotional stress. Taking care of a spouse and non-relative compared to taking care of a grandparent or parent was associated with lower emotional stress. Recipients with more comorbidities and higher care intensity were associated with worse caregiver health. Conclusions: There is a need to screen and identify caregivers who have no choice in caregiving and assist them in providing care for their recipient to prevent them from being invisible patients.