The study aimed to investigate the association between sedentary behavior and depression among rural South Africans. Data were analyzed from the cross-sectional baseline survey of the “Health and Aging in Africa: A Longitudinal Study of an INDEPTH community in South Africa (HAALSI)”. Participants responded to various measures, including sociodemographic information, health status, anthropometric measures, and sedentary behavior. The sample included 4782 persons (40 years and above). Overall, participants engaged in <4 h (55.9%), 4–<8 h (34.1%), 8–<11 h (6.4%), or 11 or more h a day (3.5%) of sedentary behavior, and 17.0% screened positive for depression. In multivariable logistic regression, which was adjusted for sociodemographic variables (Model 1) (Odds Ratio, or OR: 2.45, Confidence Interval, or CI: 1.74, 3.46) and adjusted for sociodemographic and health variables, including physical activity (Model 2) (OR: 3.00, CI: 2.00, 4.51), high sedentary time (≥11 h) was independently associated with depression. In combined analysis, compared to persons with low or moderate sedentary behavior (<8 h) and moderate or high physical activity, persons with high sedentary behavior (≥8 h) and low physical activity were more likely to have depression in Model 1 (OR: 1.60, CI: 1.65, 3.13) and Model 2 (OR: 1.60, CI: 1.05, 2.44). Findings support that sedentary behavior and combined sedentary behavior and low physical activity may be a modifiable target factor for strategies to reduce depression symptoms in this rural population in South Africa.