The dynamics of virulence evolution in vector‐borne plant pathogens can be complex. Here, we use individual‐based, quantitative‐genetic simulations to investigate how virulence evolution depends on genetic trade‐offs and population structure. Although quite generic, the model is inspired by the ecology of the plant‐pathogenic bacterium Xylella fastidiosa, and we use it to gain insights into possible modes of virulence evolution in that group. In particular, we aim to sharpen our intuition about how virulence may evolve over short time scales via antagonistically pleiotropic alleles affecting pathogen performance within hosts and vectors. We find that even when pathogens find themselves much more often in hosts than vectors, selection in the vector environment can cause correlational and potentially non‐adaptive changes in virulence in the host. The extent of such correlational virulence evolution depends on many system parameters, including the pathogen transmission rate, the proportion of the pathogen population occurring in hosts, the strengths of selection in host and vector environments, and the functional relationship between pathogen load and virulence. But there is a statistical interaction between the strength of selection in vectors and the proportion of the pathogen population in hosts, such that if within‐vector selection is strong enough, over the short term, it can dominate virulence evolution, even when the host environment predominates.