The gut microbial dysbiosis is a risk of colorectal cancer (CRC) and some bacteria have been reported as potential markers for CRC diagnosis. However, heterogeneity among studies with different populations and technologies lead to inconsistent results. Here, we investigated six metagenomic profiles of stool samples from healthy controls (HC), colorectal adenoma (CA) and CRC, and six and four genera were consistently altered between CRC and HC or CA across populations, respectively. In FengQ cohort, which composed with 61 HC, 47 CA, and 46 CRC samples, a random forest (RF) model composed of the six genera, denoted as signature-HC, distinguished CRC from HC with an area under the curve (AUC) of 0.84. Similarly, another RF model composed of the four universal genera, denoted as signature-CA, discriminated CRC from CA with an AUC of 0.73. These signatures were further validated in five metagenomic sequencing cohorts and six independent 16S rRNA gene sequencing cohorts. Interestingly, three genera overlapped in the two models (Porphyromonas, Parvimonas and Peptostreptococcus) were with very low abundance in HC and CA, but sharply increased in CRC. A concise RF model on the three genera distinguished CRC from HC or CA with AUC of 0.87 and 0.67, respectively. Functional gene family analysis revealed that Kyoto Encyclopedia of Genes and Genomes Orthogroups categories which were significantly correlated with markers in signature-HC and signature-CA were mapped into pathways related to lipopolysaccharide and sulfur metabolism, which might be vital risk factors of CRC development. Conclusively, our study identified universal bacterial markers across populations and technologies as potential aids in non-invasive diagnosis of CRC.