The detection and importance of outliers in the in vivo micronucleus assay

Abstract

Micronucleus tests are generally analysed statistically for differences between the means of treated and control groups. 'Outliers' may either be rejected or grouped together with data from less responsive animals. In either case, a valuable indicator of a small, more sensitive (responder) population sub-group may then be missed. To alleviate this problem, we have developed an additional strategy, based on historic data, for the detection of any single animal with a significant increase in micronucleated polychromatic erythrocytes in an otherwise insignificant treatment group. Forty-one sets of negative control data (of five male and five female CD-1 mice each) have been analysed. Within each set there were no significant male to female differences and data were consistent with a Poisson distribution. Pooled data from all 41 sets showed slightly hyper-Poisson variation and were adequately described by the negative binomial distribution. The negative binomial probability generating function was used to show that six or more micronuclei per 1000 polychromatic cells from one treated animal would be significant for our laboratory, methodology and strain of mouse, provided that concurrent negative control data conformed with historic values. Changes in methodology desirable for this type of analysis include increasing the number of mice in each test group and possible compensation by a reduction in the number of test groups.

Biostatistical analysis of the micronucleus mutagenicity assay based on the assumption of a mixing distribution.

“…The empirical distribution shown in Table 1 [(3) which approximates negative binomial distribution] was generated for the control groups using a PC program. Plwer estimations (based on [1,2,3];a= [0.01, 0.05, 0.10]; number ofanimals, nj= [5,10] and proportion ofnonresponders, p: [0,0.2, 0.4,0.6,0.8]. In Table 2, the power estimations of the three tests under investigations are given for nj=5.…”

“…Empirical distribution of MN. replications) of the asymptotic two-sample test based on mixed normal scores [sm(i)(d=l)], asymptotic two-sample test based on generalized MWM-score [sc(i) (a=4)], and the WMW U-test were compared for a shift alternative (mean shift between the control and dose group) with shift parameters of[1,2,3]; standard deviations: sc=1, SD=…”