A neural network approach to normality testing

Sigut, J.; Piñeiro, J. D.; Estévez, J. I.; Toledo, Pedro

doi:10.3233/ida-2006-10603

Cited by 3 publications

(31 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it is the first successful attempt at controlling the network's rate of incorrect decisions in favor of non-normality. Moreover, our network works better than the one constructed following the approach of Sigut et al [3] (which is, in turn, similar to that of Wilson and Engel [14]). Another two contributions are:…”

Section: Introductionsupporting

confidence: 60%

“…Before training the network, Wilson and Engel [14] map each sample to a vector of 16 statistics (which are not specified in the paper). Sigut et al [3] adopt a similar approach. They train a neural network on a dataset consisting of many normal and non-normal samples drawn from a large number of different distributions using the Johnson system of distributions [47,48].…”

Section: Literature Reviewmentioning

confidence: 99%

“…They train a neural network on a dataset consisting of many normal and non-normal samples drawn from a large number of different distributions using the Johnson system of distributions [47,48]. In the study of Sigut et al [3], samples can contain any number of elements, which requires a preprocessing step to transform the samples to the vectors of the same space. Sigut et al [3] map each sample to a vector of several statistics: skewness, kurtosis, the W statistic of the Shapiro-Wilk test of normality [49], the statistic of the Lin-Mudholkar test [50], the statistic of the Vasicek entropy test of normality [51], and the sample size.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Each statistical test for normality is based on a statistic that summarizes the sample being analyzed and considers only a specific property of normal distributions [1], which is why different tests sometimes give conflicting results [3]. Moreover, numerous simulation studies have shown that the tests exhibit low power (probability to identify a nonnormal distribution) when applied to samples with fewer than 100 elements [1,[4][5][6][7][8][9][10][11][12][13].…”

Section: Introductionmentioning

confidence: 99%

“…To our best knowledge, this is the third attempt at testing for normality with neural networks, the previous two being the studies of Wilson and Engel [14] and Sigut et al [3], whose work we build on. However, it is the first successful attempt at controlling the network's rate of incorrect decisions in favor of non-normality.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Testing for normality with neural networks

Simić

2021

Neural Comput & Applic

View full text Add to dashboard Cite

In this paper, we treat the problem of testing for normality as a binary classification problem and construct a feedforward neural network that can act as a powerful normality test. We show that by changing its decision threshold, we can control the frequency of false non-normal predictions and thus make the network more similar to standard statistical tests. We also find the optimal decision thresholds that minimize the total error probability for each sample size. The experiments conducted on the samples with no more than 100 elements suggest that our method is more accurate and more powerful than the selected standard tests of normality for almost all the types of alternative distributions and sample sizes. In particular, the neural network was the most powerful method for testing normality of the samples with fewer than 30 elements regardless of the alternative distribution type. Its total accuracy increased with the sample size. Additionally, when the optimal decision-thresholds were used, the network was very accurate for larger samples with 250-1000 elements. With AUROC equal to almost 1, the network was the most accurate method overall. Since the normality of data is an assumption of numerous statistical techniques, the network constructed in this study has a very high potential for use in everyday practice of statistics, data analysis and machine learning.

show abstract

Section: Introductionsupporting

confidence: 60%

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Testing for normality with neural networks

Simić

2021

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

On artificial neural networking-based process monitoring under bootstrapping using runs rules schemes

Zaman

Riaz

Ahmad

2014

Int J Adv Manuf Technol

View full text Add to dashboard Cite

Testing for Normality with Neural Networks

Simić¹

2020

Preprint

View full text Add to dashboard Cite

In this paper, we treat the problem of testing for normality as a binary classification problem and construct a feedforward neural network that can successfully detect normal distributions by inspecting small samples from them. The numerical experiments conducted on small samples with no more than 100 elements indicated that the neural network which we trained was more accurate and far more powerful than the most frequently used and most powerful standard tests of normality: Shapiro-Wilk, Anderson-Darling, Lilliefors and Jarque-Berra, as well as the kernel tests of goodness-of-fit. The neural network had the AUROC score of almost 1, which corresponds to the perfect binary classifier. Additionally, the network's accuracy was higher than 96% on a set of larger samples with 250 − 1000 elements. Since the normality of data is an assumption of numerous techniques for analysis and inference, the neural network constructed in this study has a very high potential for use in everyday practice of statistics, data analysis and machine learning in both science and industry.

show abstract

A neural network approach to normality testing

Cited by 3 publications

References 14 publications

Testing for normality with neural networks

Testing for normality with neural networks

On artificial neural networking-based process monitoring under bootstrapping using runs rules schemes

Testing for Normality with Neural Networks

Contact Info

Product

Resources

About