Use of genetic methods to estimate effective population size (N e ) is rapidly increasing, but all approaches make simplifying assumptions unlikely to be met in real populations. In particular, all assume a single, unstructured population, and none has been evaluated for use with continuously distributed species. We simulated continuous populations with local mating structure, as envisioned by Wright's concept of neighborhood size (NS), and evaluated performance of a single-sample estimator based on linkage disequilibrium (LD), which provides an estimate of the effective number of parents that produced the sample (N b ). Results illustrate the interacting effects of two phenomena, drift and mixture, that contribute to LD. Samples from areas equal to or smaller than a breeding window produced estimates close to the NS. As the sampling window increased in size to encompass multiple genetic neighborhoods, mixture LD from a two-locus Wahlund effect overwhelmed the reduction in drift LD from incorporating offspring from more parents. As a consequence,N b never approached the global N e , even when the geographic scale of sampling was large. Results indicate that caution is needed in applying standard methods for estimating effective size to continuously distributed populations. Heredity (2013) 111, 189-199; doi:10.1038/hdy.2013; published online 8 May 2013Keywords: genetic monitoring; inbreeding; isolation-by-distance; linkage disequilibrium; wahlund effect; wright's neighborhood
INTRODUCTIONThe concept of effective population size (N e ) provides a way to quantify the evolutionary changes caused by random processes in finite populations (Charlesworth, 2009). Since its formal definition by Wright (1931) as the size of an ideal population that has played the same rate of genetic drift as an actual population of interest, N e has a central role in studies of evolution and ecology. In addition to directly affecting the rates of loss of neutral genetic variability and frequency change of neutral alleles, N e mediates the effectiveness of migration and selection, which are predictable in large populations but can be overwhelmed by drift in small ones.The original concept of N e envisaged a single population or a series of semi-discrete populations connected by limited migration. In many species, however, individuals are distributed more or less continuously across large landscapes, and matings occur more frequently among spatially proximal individuals. This type of demography produces a pattern of 'isolation-by-distance' (IBD; Wright, 1943), in which genetic differentiation increases with distance, but there are no distinct breaks or discrete subunits within the global population. Species distributed in this fashion pose a particular challenge in applying the concept of effective population size. For example, recent research using the coalescent (for example, Barton and Wilson, 1995;Wilkins, 2004) has shown that evolutionary processes in continuously distributed populations exhibit both a short-term dynamic that is strong...