Physical activity (PA) is a known behavior to reduce cancer risk and improve cancer survivorship, yet adherence to PA guidelines is poor among the general population and cancer survivors. The purpose of this study was to determine the extent to which patients referred for exercise consultation within a clinical cancer prevention setting were meeting aerobic and resistance physical activity (PA) guidelines and to identify factors associated with guideline adherence. Between 2013 and 2015, cancer prevention patients and cancer survivors were interviewed by an exercise physiologist within an Integrative Health Program at The University of Texas MD Anderson Cancer Prevention Center. PA adherence was defined as at least 150-minutes of moderate-intensity or 75-minutes of vigorous-intensity PA per week, along with resistance training at least 2 days per week. Logistic regression was used to determine factors associated with meeting or not meeting PA guidelines for aerobic exercise, resistance exercise, and aerobic and resistance exercise combined. Among 1,024 cancer prevention patients and survivors, 9% of patients adhered to guideline-based PA. Adherence to aerobic and resistance guidelines were 20% and 12%, respectively. Overweight or obesity was associated with not meeting guideline-based PA in both cancer prevention patients and cancer survivors. Among breast cancer survivors, combination treatment with surgery, radiation, and chemotherapy (‘multimodal therapy’) was robustly associated with not meeting aerobic guidelines (OR 2.20, 95% CI: 1.17 to 4.16). BMI and breast cancer treatment history are key determinants of PA behavior among cancer prevention patients and survivors. Poor adherence to PA guidelines is a key issue for cancer prevention patients and survivors, particularly obese patients and women who receive multimodal therapy for breast cancer. Identifying and connecting patients at highest risk of poor PA adherence with exercise programs is needed to improve PA, a key modifiable cancer risk factor.