Small populations are predicted to have reduced capacity to adapt to environmental change for two reasons. First, population genetic models indicate that genetic variation and potential response to selection should be positively correlated with population size. The empirical support for this prediction is mixed: DNA markers usually reveal low heterozygosity in small populations, whereas quantitative traits show reduced heritability only in the smallest and most inbred populations. Quantitative variation can even increase in bottlenecked populations although this effect seems unlikely to increase the adaptive potential of populations. Second, individuals in small populations have lower fitness owing to environmental stress and genetic problems such as inbreeding, which can substantially increase the extinction probability of populations in changing environments. This second reason has not been included in assessments of critical population size assuring evolvability and makes it likely that many small threatened populations have a decreased potential for adaptation.

Changes in environmental conditions can rapidly shift allele frequencies in populations of species with relatively short generation times. Frequency shifts might be detectable in neutral genetic markers when stressful conditions cause a population decline. However, frequency shifts that are diagnostic of specific conditions depend on isolating sets of genes that are involved in adaptive responses. Shifts at candidate loci underlying adaptive responses and DNA regions that control their expression have now been linked to evolutionary responses to pollution, global warming and other changes. Conversely, adaptive constraints, particularly in physiological traits, are recognized through DNA decay in candidate genes. These approaches help researchers and conservation managers understand the power and constraints of evolution.

Why species have geographically restricted distributions is an unresolved question in ecology and evolutionary biology. Here, we test a new explanation that mutation accumulation due to small population size or a history of range expansion can contribute to restricting distributions by reducing population growth rate at the edge. We examined genomic diversity and mutational load across the entire geographic range of the North American plant Arabidopsis lyrata, including old, isolated populations predominantly at the southern edge and regions of postglacial range expansion at the northern and southern edges. Genomic diversity in intergenic regions declined toward distribution edges and signatures of mutational load in exon regions increased. Genomic signatures of mutational load were highly linked to phenotypically expressed load, measured as reduced performance of individual plants and lower estimated rate of population growth. The geographic pattern of load and the connection between load and population growth demonstrate that mutation accumulation reduces fitness at the edge and helps restrict species' distributions.

Cross-fertilisation predominates in eukaryotes, but shifts to self-fertilisation are common and ecologically and evolutionarily important. Reproductive assurance under outcross gamete limitation is one eco-evolutionary process held responsible for the shift to selfing. Although small effective population size is a situation where selfing plants could theoretically benefit from reproductive assurance, empirical tests of the role of population size are rare. Here, we show that selfing evolved repeatedly at range margins, where historical demographic processes produced low effective population sizes. Outcrossing populations of North American Arabidopsis lyrata have low genetic diversity at geographic margins, with a signature of post-glacial range expansion in the north and rear-edge isolation in the south. Selfing populations occur at the margins of two genetic groups and never in their interior. These results corroborate small effective population size as the promoter of self-fertilisation and have important implications for our understanding of species turnover, range limits and range dynamics.

Reduced genetic variation at marker loci in small populations has been well documented, whereas the relationship between quantitative genetic variation and population size has attracted little empirical investigation. Here we demonstrate that both neutral and quantitative genetic variation are reduced in small populations of a fragmented plant metapopulation, and that both drift and selective change are enhanced in small populations. Measures of neutral genetic differentiation (FST) and quantitative genetic differentiation (QST) in two traits were higher among small demes, and QST between small populations exceeded that expected from drift alone. This suggests that fragmented populations experience both enhanced genetic drift and divergent selection on phenotypic traits, and that drift affects variation in both neutral markers and quantitative traits. These results highlight the need to integrate natural selection into conservation genetic theory, and suggests that small populations may represent reservoirs of genetic variation adaptive within a wide range of environments.