Infection of plants by viruses is a complex process involving several steps: inoculation into plant cells, replication in inoculated cells and plant colonization. The success of the different steps depends, in part, on the viral effective population size (Ne), defined as the number of individuals passing their genes to the next generation. During infection, the virus population will undergo bottlenecks, leading to drastic reductions in Ne and, potentially, to the loss of the fittest variants. Therefore, it is crucial to better understand how plants affect Ne. We aimed to (i) identify the plant genetic factors controlling Ne during inoculation, (ii) understand the mechanisms used by the plant to control Ne and (iii) compare these genetic factors with the genes controlling plant resistance to viruses. Ne was measured in a doubled-haploid population of Capsicum annuum. Plants were inoculated with either a Potato virus Y (PVY) construct expressing the green fluorescent protein or a necrotic variant of Cucumber mosaic virus (CMV). Newas assessed by counting the number of primary infection foci on cotyledons for PVY or the number of necrotic local lesions on leaves for CMV. The number of foci and lesions was correlated (r=0.57) and showed a high heritability (h2=0.93 for PVY and h2=0.98 for CMV). The Ne of the two viruses was controlled by both common quantitative trait loci (QTLs) and virus-specific QTLs, indicating the contribution of general and specific mechanisms. The PVY-specific QTL colocalizes with a QTL that reduces PVY accumulation and the capacity to break down a major-effect resistance gene.