Listeria monocytogenes (Lm) is a ubiquitous bacterium that causes the serious foodborne illness listeriosis. Although soil is a primary reservoir and a central habitat for Lm, little information is available on the genetic features underlying the fitness of Lm strains in this complex habitat. The aim of this study was to identify (i) correlations between the strains fitness in soil, their origin and their phylogenetic position (ii) identify genetic markers allowing Lm to survive in the soil. To this end, we assembled a balanced panel of 216 Lm strains isolated from three major ecological compartments (outdoor environment, animal hosts, and food) and from 33 clonal complexes occurring worldwide. The ability of the 216 strains to survive in soil was tested phenotypically. Hierarchical clustering identified three phenotypic groups according to the survival rate (SR): phenotype 1 “poor survivors” (SR < 2%), phenotype 2 “moderate survivors” (2% < SR < 5%) and phenotype 3 “good survivors” (SR > 5%). Survival in soil depended neither on strains’ origin nor on their phylogenetic position. Genome-wide-association studies demonstrated that a greater number of genes specifically associated with a good survival in soil was found in lineage II strains (57 genes) than in lineage I strains (28 genes). Soil fitness was mainly associated with variations in genes (i) coding membrane proteins, transcription regulators, and stress resistance genes in both lineages (ii) coding proteins related to motility and (iii) of the category “phage-related genes.” The cumulative effect of these small genomic variations resulted in significant increase of soil fitness.