Prostate motion (standard deviation, range of motion, and diffusion coefficient) was calculated from 4D ultrasound data of 1791 fractions of radiation therapy in N = 100 patients. The inner diameter of the lesser pelvis was obtained from transversal slices through the pubic symphysis in planning CTs. On the lateral and craniocaudal axes, motility increases significantly (t-test, p < 0.005) with the inner diameter of the lesser pelvis. A diameter of >106 mm (ca. 6th decile) is a good predictor for high prostate intrafraction motion (ca. 9th decile). The corresponding area under the receiver operator curve (AUROC) is 80% in the lateral direction, 68% to 80% in the craniocaudal direction, and 62% to 70% in the vertical direction. On the lateral x-axis, the proposed test is 100% sensitive and has a 100% negative predictive value for all three characteristics (standard deviation, range of motion, and diffusion coefficient). On the craniocaudal z-axis, the proposed test is 79% to 100% sensitive and reaches 95% to 100% negative predictive value. On the vertical axis, the proposed test still delivers 98% negative predictive value but is not particularly sensitive. Overall, the proposed predictor is able to help identify patients at risk of high prostate motion based on a single planning CT.