Increasing physical activity and decreasing sedentary behavior are associated with a higher quality of life and lower mortality rates for cancer survivors, a growing population group. Studies detailing the behavior of cancer survivors are limited. Therefore, we investigated physical activity and sedentary behavior of cancer survivors using data from the National Health and Nutrition Examination Survey (NHANES) 2007–2010. Participants were those who provided physical activity and sedentary behavior data. Those who were pregnant, <20 years old, or <3 years from their cancer diagnosis were excluded. A cancer case was a self-reported diagnosis by a physician. We identified 741 cancer survivors and 10,472 non-cancer participants. After adjustment for age, race, gender, education status, body mass index, and smoking status, cancer survivors (n = 10,472) reported significantly longer duration of sedentary behavior (OR = 1.42, 95% CI (1.12, 1.80) for 8 or more hours, p-value for trend = 0.09), compared to non-cancer participants (n = 741). They also reported non-significant increases in maximum intensity, duration, frequency, and energy expenditure, whereas they reported significant increases in moderate intensity (OR = 1.26, 95% CI (1.01, 1.57)), moderate frequency (1–4 times/week) (OR = 1.32, 95% CI (1.00, 1.74)), and moderate energy expenditure (4018.5–7623.5 kcal) (OR = 1.30, 95% CI (1.00, 1.71)) of physical activity, compared to non-cancer participants. These patterns are similar for breast and prostate cancer survivors, with prostate cancer survivors more likely to engage in physical activity for more than one hour per day (OR = 1.98, 95% CI (1.05, 3.71)). Our findings suggest that cancer survivors tend to have more physical activity, but they are also more likely to engage in sedentary behavior.