In a cross-sectional study conducted four years ago to assess the validity of the Brazilian version of the Eating Attitudes for the identification of abnormal eating behaviors in a population of young females in Southern Brazil, 56 women presented abnormal eating behavior as indicated by the EAT-26 and the Edinburgh Bulimic Investigation Test. They were each matched for age and neighborhood to two normal controls (N = 112) and were re-assessed four years later with the two screening questionnaires plus the Composite International Diagnostic Interview (CIDI). The EAT results were then compared to diagnoses originating from the CIDI. To evaluate the temporal stability of the two screening questionnaires, a test-retest design was applied to estimate kappa coefficients for individual items. Given the prevalence of eating disorders of 6.2%, the CIDI psychiatry interview was applied to 161 women. Of these, 0.6% exhibited anorexia nervosa and 5.6%, bulimia nervosa (10 positive cases). The validity coefficients of the EAT were: 40% sensitivity, 84% specificity, and 14% positive predictive value. Cronbach's coefficient was 0.75. For each EAT item, the kappa index was not higher than 0.344 and the correlation coefficient was lower than 0.488. We conclude that the EAT-26 exhibited low validity coefficients for sensitivity and positive predictive value, and showed a poor temporal stability. It is reasonable to assume that these results were not influenced by the low prevalence of eating disorders in the community. Thus, the results cast doubts on the ability of the EAT-26 test to identify cases of abnormal eating behaviors in this population.