Kelp forests are declining in many parts of the northeast Pacific. In small populations, genetic drift can reduce adaptive variation and increase fixation of recessive deleterious alleles, but natural selection may purge harmful variants. To understand evolutionary dynamics and inform restoration strategies, we investigated genetic structure and the outcomes of genetic drift and purging by sequencing the genomes of 429 bull kelp (Nereocystis luetkeana) and 211 giant kelp (Macrocystis pyrifera sensu lato) from the coastlines of British Columbia and Washington. We identified 6-7 geographically and genetically distinct clusters in each species. Low effective population size was associated with low genetic diversity and high inbreeding coefficients (including increased selfing rates), with extreme variation in these genetic health indices among bull kelp populations but more moderate variation in giant kelp. We found no evidence that natural selection is purging putative recessive deleterious alleles in either species. Instead, genetic drift has fixed many such alleles in small populations of bull kelp, leading us to predict (1) reduced within-population inbreeding depression in small populations, which may be associated with an observed shift toward increased selfing rate, and (2) hybrid vigour in crosses between small populations. Our genomic findings imply several strategies for optimal sourcing and crossing of populations for restoration and aquaculture, but which require experimental validation. Overall, our work reveals strong genetic structure and suggests that conservation strategies should consider the multiple health risks faced by small populations whose evolutionary dynamics are dominated by genetic drift.