Marital status mediates an association between physical activity (PA) and TV time with health outcomes. However, population-based studies have revealed that the health effect of marriage or divorce is age-dependent and differs between women and men. The study aimed to identify the age and sex-related associations between marital status with PA and TV time. We used data from Vigitel, an annual telephone survey started in 2006 in Brazil. We applied a complex sample logistic regression model to estimate the odds for PA and TV time comparing marital statuses according to age and sex subgroups, independent of obesity, hypertension, diabetes, self-assessed poor health, and smoking. Our sample included 561,837 individuals from 18 to 99 years, with a TV time > 3 h/day (prevalence = 25.2%) and PA > 150 min/week (prevalence = 35%). Later, we divided our sample in seven age groups by marital status and sex. Compared to single individuals, married men and women were less likely to watch TV more than 3 h/day in participants >30 years old. When compared to single, married participants were less likely to do more than 150 min of PA/week at younger age groups. Married women older than 40 years were more likely to do more than 150 min of PA/week than the single ones, while there were no differences among married men by age group. In conclusion, our study suggests that the investments in public policies to encourage the practice of PA and reduction of TV time could be based on the marital status, sex, and age, prioritizing less active groups.