Few studies have investigated food security, dietary quality, or unfavorable food intake through self-reports among children and adolescents in Asia. This study assessed the association of perceived food insecurity with dietary quality and unfavorable food intake among 1111 children and 538 adolescents from economically disadvantaged families in Taiwan. Food security status was collected by a validated questionnaire. Dietary quality was examined using a dietary diversity score (DDS). Unfavorable food intake was defined as fried food, bread/cake/pastries, sugar-sweetened beverages, and biscuits/chips. Food-insecure participants had lower DDS, whole grains and protein-rich food intake than food-secure participants. Furthermore, food-insecure children had a higher frequency of unfavorable food consumption. The level of children’s food insecurity was inversely associated with DDS (β: −0.047, 95% CI: −0.085 to −0.009) but positively with bread/pastry (β: 0.103, 95% CI: 0.022–0.184) and sugar-sweetened beverages (β: 0.117, 95% CI: 0.018–0.215) intake. Adolescents who reported food insecurity and not enough money for household expenses had an odds ratio of 2.85 (95% CI: 1.15–7.10) for poor DDS relative to their food-secure and financially able counterparts. We recommended that health policy needs to include diversifying food and nutrition education for vulnerable children and adolescents to improve dietary quality.