Several recent studies published in Poultry Science employed sampling strategies designed to include only birds near the pen mean. These nonrandom samples were used to test hypotheses concerning the effect of treatment on characteristics of interest. In this study, 415,000 experiments were simulated to determine the effects of systematically eliminating data to concentrate observations around the pen mean. Rejection rates of the true null hypotheses were compared to theoretical values. These results show that such nonrandom sampling schemes lead to significance levels that are overstated by a factor of four or more. (