Objective: To explore a new predictive model of lymphatic vascular infiltration (LVI) in rectal cancer based on magnetic resonance (MR) and computed tomography (CT). Methods: A retrospective study was conducted on 94 patients with histologically confirmed rectal cancer, they were randomly divided into training cohort (n = 65) and validation cohort (n = 29). All patients underwent MR and CT examination within 2 weeks before treatment. On each slice of the tumor, we delineated the volume of interest on T2-weighted imaging, diffusion weighted imaging, and enhanced CT images, respectively. A total of 1,188 radiological features were extracted from each patient. Then, we used the student t-test or Mann-Whitney U-test, Spearman's rank correlation and least absolute shrinkage and selection operator (LASSO) algorithm to select the strongest features to establish a single and multimodal logic model for predicting LVI. Receiver operating characteristic (ROC) curves and calibration curves were plotted to determine how well they explored LVI prediction performance in the training and validation cohorts. Results: An optimal multi-mode radiology nomogram for LVI estimation was established, which had significant predictive power in training (AUC, 0.884; 95% CI, 0.803-0.964) and validation (AUC, 0.876; 95% CI, 0.721-1.000). Calibration curve and decision curve analysis showed that the multimodal radiomics model provides greater clinical benefits. Conclusion: Multimodal (MR/CT) radiomics models can serve as an effective visual prognostic tool for predicting LVI in rectal cancer. It demonstrated great potential of preoperative prediction to improve treatment decisions.