Extreme climatic events and variability are on the rise around the world, with varying implications for populations across socio-economic conditions. Effective strategies for climate adaptation and development depend on understanding these differential sensitivities to climatic variability. This study focuses on a vulnerable population living in forest-fringe villages of central India, where seasonal migration is a common livelihood strategy for poor households to supplement their incomes with remittances. We quantify the relative sensitivity of a decision to migrate for the first time to climate and socio-economic variables and how the sensitivities vary for different segments of the population. We surveyed 5000 households in 500 forest-fringe villages to identify patterns of migration from 2013 to 2017. Using a mixed-effects logistic regression model, we predicted the probability of first-time migration of a household member based on climate variables and household- and district-level characteristics. We find that households in more agricultural and prosperous districts experience lower rates of migration but are more sensitive to climatic variability than households in poorer districts. The probability of first-time migration from a household in the most prosperous district increases by approximately 40% with one standard deviation in mean maximum temperature or rainfall from the 1981–2017 mean. However, the probability of migration does not vary as a function of climatic variability for households in the poorest district. We attribute this difference in sensitivities to the greater dependence on agriculture and irrigation in more prosperous districts and poverty-driven dependence on migration regardless of the climate in poorer districts. Households investing remittances from migration in agricultural intensification could become increasingly sensitive to climate variability, particularly with water shortages and projected increases in climate variability in the region. Promotion of non-agricultural livelihood options and climate-resilient agriculture could the reduce sensitivity of migration to climate variability in the study region.