The ability to metabolize sucrose is a variable trait within the family Vibrionaceae. The marine bacterium Photobacterium damselae subsp. damselae (Pdd), pathogenic for marine animals and humans, is generally described as negative for sucrose utilization (Scr−). Previous studies have reported sucrose-utilizing isolates (Scr+), but the genetic basis of this variable phenotype remains uncharacterized. Here, we carried out the genome sequencing of five Scr+ and two Scr− Pdd isolates and conducted a comparative genomics analysis with sixteen additional Pdd genomes sequenced in previous studies. We identified two different versions of a four-gene cluster (scr cluster) exclusive of Scr+ isolates encoding a PTS system sucrose-specific IIBC component (scrA), a fructokinase (scrK), a sucrose-6-phosphate hydrolase (scrB), and a sucrose operon repressor (scrR). A scrA deletion mutant did not ferment sucrose and was impaired for growth with sucrose as carbon source. Comparative genomics analyses suggested that scr clusters were acquired by horizontal transfer by different lineages of Pdd and were inserted into a recombination hot-spot in the Pdd genome. The incongruence of phylogenies based on housekeeping genes and on scr genes revealed that phylogenetically diverse gene clusters for sucrose utilization have undergone extensive horizontal transfer among species of Vibrio and Photobacterium.