Many pathogenic bacteria are recognized as species complexes and uncertainties regarding the organization of their genetic diversity are challenges for research efforts. Within Agrobacterium tumefaciens, multiple genomospecies have been identified; however, the exact species boundaries are unclear, which causes chaos in nomenclature and hampers communication. In this work, we conducted targeted genome sequencing to achieve a comprehensive and balanced taxon sampling within this complex. Our results from genome-wide sequence identity, core genome phylogeny, and gene content not only supported that those recognized genomospecies are distinct biological entities but also identified novel genomospecies. Based on the fully resolved phylogeny, we further investigated the evolution of genes critical in Agrobacterium fitness and ecology. For the type VI secretion system (T6SS) involved in interbacterial competition, multiple losses and one horizontal gene transfer (HGT) event were inferred. For the tumor-inducing plasmids (pTi) and the pTi-encoded type IV secretion system (T4SS) that determine Agrobacterium phytopathogenicity, the evolution of these accessory replicons was decoupled from the chromosomes, thus contributing to another level of complexity. Intriguingly, for both T6SS and T4SS, genes that encode the structural components are highly conserved, whereas extensive diversity exists at multiple levels (i.e., between-species, within-species, intra-genome, and intra-gene) for genes that encode effectors and associated proteins. These findings suggest that opposite modes of selection may act on components conferring different functions within a system. In conclusion, this work provides insights into the genomic diversification of these bacteria and sheds light on the modularity of their molecular evolution.