Background Sedentary behaviors are prevalent in Chinese children, however, the studies on their trends and socioeconomic disparities are scarce. We examined the time trends of daily television (TV) viewing and video game playing and the associated socioeconomic factors in Chinese children in Hong Kong, the most developed and westernized city in China. Methods In a panel data study involving 538,300 primary four and 510,294 primary six students from 1999/2000 to 2008/09, data on socioeconomic status, sedentary behaviors (TV viewing and video game playing) and other lifestyle habits were collected using a self-administered questionnaire. Trends in sedentary behaviors over time were assessed. Their socioeconomic disparities were examined by interactions in generalized estimating equations with the adjustment for weight status and extracurricular physical activities. Results The age and sex-standardized prevalence of ≥2 h daily TV viewing decreased from 51.4% (95% confidence interval [CI] 51.1–51.8%) in 1999/2000 to 43.8% (95% CI 43.4–44.2%) in 2008/09 (P for trend < 0.001), whereas that of ≥1 h daily video game playing increased from 8.2% (95% CI 7.9–8.4%) to 22.4% (95% CI 22.0–22.7%). Both sedentary behaviors were more prevalent in boys than girls, but the disparities decreased over time (Ratio of odds ratio [ROR] = 0.996 and 0.924 for TV viewing and video game playing, respectively). In contrast, both sedentary behaviors were increasingly more prevalent in children whose parents had lower education levels or non-managerial/professional occupations (ROR 1.006–1.082). Conclusions Children in lower socioeconomic families in Hong Kong were increasingly at risk of having sedentary behaviors over years and thus deserve more attention. Effective strategies targeting children and/or their parents of lower socioeconomic status are needed to reduce sedentary behaviors.