Objective. This study aimed to explore the genes regulating lymph node metastasis in colorectal cancer (CRC) and to clarify their relationship with tumor immune cell infiltration and patient prognoses. Methods. The data sets of CRC patients were collected through the Cancer Gene Atlas database; the differentially expressed genes (DEGs) associated with CRC lymph node metastasis were screened; a protein–protein interaction (PPI) network was constructed; the top 20 hub genes were selected; the Gene Ontology functions and the Kyoto Encyclopedia of Genes and Genomes pathways were enriched and analyzed. The Least Absolute Shrinkage and Selection Operator (LASSO) regression method was employed to further screen the characteristic genes associated with CRC lymph node metastasis in 20 hub genes, exploring the correlation between the characteristic genes and immune cell infiltration, conducting a univariate COX analysis on the characteristic genes, obtaining survival-related genes, constructing a risk score formula, conducting a Kaplan–Meier analysis based on the risk score formula, and performing a multivariate COX regression analysis on the clinical factors and risk scores. Results. A total of 62 DEGs associated with CRC lymph node metastasis were obtained. Among the 20 hub genes identified via PPI, only calcium-activated chloride channel regulator 1 (CLCA1) expression was down-regulated in lymph node metastasis, and the rest were up-regulated. A total of nine characteristic genes associated with CRC lymph node metastasis (KIF1A, TMEM59L, CLCA1, COL9A3, GDF5, TUBB2B, STMN2, FOXN1, and SCN5A) were screened using the LASSO regression method. The nine characteristic genes were significantly related to different kinds of immune cell infiltration, from which three survival-related genes (TMEM59L, CLCA1, and TUBB2B) were screened. A multi-factor COX regression showed that the risk scores obtained from TMEM59L, CLCA1, and TUBB2B were independent prognostic factors. Immunohistochemical validation was performed in tissue samples from patients with rectal and colon cancer. Conclusion. TMEM59L, CLCA1, and TUBB2B were independent prognostic factors associated with lymphatic metastasis of CRC.