Integrated crop and livestock systems (ICLSs) have been considered an important management-based decision to improve soil health by carbon sequestration. A two-year study (2019–2021) at CPBES in Newton, MS, was conducted to evaluate the effect of an ICLS on soil microbial diversity in the southeastern region of the USA, representing agroclimatic conditions that are warm and humid. Amplicons targeting bacterial 16S rRNA genes and fungal ITS2 regions were sequenced. Taxonomic assignment and characterization of microbial diversity were performed using QIIME2®. Soil fungal diversity pattern showed significant difference (alpha diversity, p = 0.031 in 2020 and beta diversity, p = 0.037 in 2021). In contrast, no significant differences were observed in bacterial diversity. However, there were several beneficial bacterial phyla, such as Proteobacteria and Actinobacteria, and fungal phyla such as Ascomycota, which were dominant in both years and did not show significant differences due to cover crop treatments. Canonical Correspondence Analysis (CCA) and Mantel test showed significant influence on fungal diversity due to carbon (rm = 0.2581, p = 0.022), nitrogen (rm = 0.2921, p = 0.0165), and electrical conductivity (rm = 0.1836, p = 0.0583) in 2021, and on bacterial diversity due to EE-GRSP (rm = 0.22, p = 0.02) in 2020. However, the results showed that there were no significant differences between the cover crop treatments that were consistent over a two-year study period. However, the mix of different cover crops such as oats (Avena sativa L.), crimson clover (Trifolium incarnatum L.), and tillage radish (Raphanus sativus L.) demonstrated higher positive correlation and lower negative correlation with different bacterial and fungal phyla. Long term study of ICLS is suggested to understand the shift in microbiome that would help in understanding the role of cover crops and grazing in improving crop production sustainably.