Human-driven habitat fragmentation and loss have led to a proliferation of small and isolated plant and animal populations with high risk of extinction. One of the main threats to extinction in these populations is inbreeding depression, which is primarily caused by recessive deleterious mutations becoming homozygous due to inbreeding. The typical approach for managing these populations is to maintain high genetic diversity, increasingly by translocating individuals from large populations to initiate a “genetic rescue.” However, the limitations of this approach have recently been highlighted by the demise of the gray wolf population on Isle Royale, which declined to the brink of extinction soon after the arrival of a migrant from the large mainland wolf population. Here, we use a novel population genetic simulation framework to investigate the role of genetic diversity, deleterious variation, and demographic history in mediating extinction risk due to inbreeding depression in small populations. We show that, under realistic models of dominance, large populations harbor high levels of recessive strongly deleterious variation due to these mutations being hidden from selection in the heterozygous state. As a result, when large populations contract, they experience a substantially elevated risk of extinction after these strongly deleterious mutations are exposed by inbreeding. Moreover, we demonstrate that, although genetic rescue is broadly effective as a means to reduce extinction risk, its effectiveness can be greatly increased by drawing migrants from small or moderate-sized source populations rather than large source populations due to smaller populations harboring lower levels of recessive strongly deleterious variation. Our findings challenge the traditional conservation paradigm that focuses on maximizing genetic diversity in small populations in favor of a view that emphasizes minimizing strongly deleterious variation. These insights have important implications for managing small and isolated populations in the increasingly fragmented landscape of the Anthropocene.