Necessary Condition Analysis: Type I Error, Power, and Over-Interpretation of Test Results. A Reply to a Comment on NCA. Commentary: Predicting the Significance of Necessity

Dul, Jan; Laan, E. van der; Kuik, Roelof; Karwowski, Maciej

doi:10.3389/fpsyg.2019.01493

Cited by 12 publications

(18 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, consistency was not consistent with a substantial necessarybut-not-sufficient condition for grades, d = .11, p = .34 (see Table 2 and Figure 4), although the significance and effect size estimates presented may also be influenced by the modest statistical power of this study. Sample size was limited by class size, and while the NCA significance test is powerful enough to rule out an effect being the product of randomness (Dul et al, 2018(Dul et al, , 2019, as with traditional regression-based and correlational analyses, NCA conducted with a small sample is not immune to Type II error.…”

Section: Resultsmentioning

confidence: 99%

“…To guard against NCA effects being the result of empty space produced by unrelated variables, regardless of skew, Dul and colleagues developed a permutation test of statistical significance for NCA (p, Dul et al, 2018;Dul, van der Laan, Kuik, & Karwowski, 2019). The p value effectively tests the probability that empty space, and the resultant magnitude of d, is a random result of unrelated variables (Dul, 2019;Dul et al, 2018).…”

Section: Methodsological Considerationsmentioning

confidence: 99%

See 1 more Smart Citation

Are individual characteristics and behaviors necessary-but-not-sufficient conditions for academic success?: A demonstration of Dul's (2016) necessary condition analysis

Tynan

Credé

Harms

2020

Learning and Individual Differences

View full text Add to dashboard Cite

Claiming that high levels of an independent variable represent a necessary-but-not-sufficient condition for an outcome suggests that the outcome is only possible -but not guaranteed -with high levels of that variable. Necessary condition analysis (NCA) allows researchers to determine if an observed relation between an independent variable and a dependent variable is consistent with such a necessary-but-notsufficient relation. Using both archival and primary data, we apply Dul's ( 2016) necessary condition analysis techniques to common correlates of academic success in college. We find data patterns that are consistent with necessary-but-not-sufficient conditions for academic success for a variety of variables including class attendance, grit-perseverance, growth mindset, prior achievement, and admissions test scores. Our findings imply that some individual characteristics and behaviors may constrain the level of grades possible in college and that researchers may benefit from considering necessity models of academic performance. We discuss further applications of necessary condition analysis in educational research as a supplement to traditional data analysis.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsological Considerationsmentioning

confidence: 99%

Are individual characteristics and behaviors necessary-but-not-sufficient conditions for academic success?: A demonstration of Dul's (2016) necessary condition analysis

Tynan

Credé

Harms

2020

Learning and Individual Differences

View full text Add to dashboard Cite

show abstract

“…There are two steps in NCA ( Dul et al, 2019 ), determining ceiling lines and bottleneck tables are the first. Unlike traditional regression models where a line is drawn through the middle of the data in an XY-plot, a ceiling line is created in NCA.…”

Section: Methodsmentioning

confidence: 99%

The Importance of Well-Being on Resiliency of Filipino Adults During the COVID-19 Enhanced Community Quarantine: A Necessary Condition Analysis

Camitan

Bajin²

2021

Front. Psychol.

View full text Add to dashboard Cite

Nation-wide community quarantines and social distancing are part of the new normal because of the global COVID-19 pandemic. Since extensive and prolonged lockdowns are relatively novel experiences, not much is known about the well-being of individuals in such extreme situations. This research effort investigated the relationship between well-being elements and resiliency of 533 Filipino adults who were placed under the nationwide enhanced community quarantine (ECQ) during the COVID-19 pandemic. Participants comprised of 376 females (70.56%) and 157 males (29.45%). The median and mode ages of the participants is 23 years, while 25 is the mean age. PERMA Profiler was used to measure participants’ well-being elements, while Connor-Davidson Resiliency Scale-10 (CD-RISC-10) was used to measure their resiliency. Collected data were analyzed using the regression model and necessary condition analysis. This study corroborated that all the five pillars of well-being are significant positive correlates of resiliency (p < 0.00) in quarantined adults. The results shown accomplishment (β = 0.447, p < 0.01) positively predicts resiliency, while negative emotions (β = −0.171, p < 0.00) negatively predict resiliency. Lastly, the five pillars of well-being are necessary-but-not-sufficient conditions (ceiling envelopment with free disposal hull, CE-FDH p < 0.00) of resiliency. Our results cast a new light on well-being elements as constraints rather than enablers of resiliency. This novel result shows that optimum resiliency is only possible when all the five pillars of well-being are taken care of and when a person is at least minimally contented with their physical health. The present findings underscore the importance of a holistic as against an atomistic approach to maintaining good mental health, which suggests that deficiencies in certain areas of well-being may not be fully addressed by overcompensating on other areas, as all five pillars of well-being are necessary-but-not-sufficient conditions of resiliency. The study ends with the recommendation for the use of necessary condition analysis to study both classical and novel psychological research problems.

show abstract

“…We suspect that Camitan and Bajin, and maybe also others, may have been duped by the name of the test as well as the content of Dul's [1] original paper (including its title) into believing that NCA analyzes if a condition X can be assumed to be necessary, or even necessary but not sufficient, for an outcome Y. However, as later recognized by the developers, NCA is only capable of evaluating if the association between X and Y is characterized by some unspecified type of non-randomness [7]. With this in mind, it might be asked if it would not be appropriate to rename the analysis "a randomness test (ART)" or something similar.…”

Section: Discussionmentioning

confidence: 99%

“…The developers of NCA have acknowledged that NCA does not assess if X can be assumed to be necessary for Y, and even less if X can be assumed to be necessary but not sufficient for Y. Instead, NCA has been described as a null hypothesis test and a significant finding only indicates that the association between X and Y is characterized by some unspecified type of nonrandomness [7]. However, NCA seems inferior to ordinary linear regression analysis for detecting nonrandom associations [8].…”

Section: Introductionmentioning

confidence: 99%

Conclusions about well-being being necessary but not sufficient for resilience may be unwarranted

Sorjonen¹,

Melin²

2021

Preprint

View full text Add to dashboard Cite

After conducting necessary condition analysis (NCA), researchers have concluded that a certain, not too low, level of well-being is necessary but not sufficient for a high level of resilience. However, as acknowledged by the developers of the test, NCA only evaluates if the association between two variables is characterized by some unspecified type of non-randomness and not conditions of necessity. The present findings, based on re-analyses of data, indicate a significant necessity effect of resilience on overall well-being, which is not logically compatible with well-being being necessary but not sufficient for resilience. Analyses with an extended version of NCA suggest that the association between overall well-being and resilience is characterized by equal degrees of necessity and sufficiency.

show abstract

Necessary Condition Analysis: Type I Error, Power, and Over-Interpretation of Test Results. A Reply to a Comment on NCA. Commentary: Predicting the Significance of Necessity

Cited by 12 publications

References 4 publications

Are individual characteristics and behaviors necessary-but-not-sufficient conditions for academic success?: A demonstration of Dul's (2016) necessary condition analysis

Are individual characteristics and behaviors necessary-but-not-sufficient conditions for academic success?: A demonstration of Dul's (2016) necessary condition analysis

The Importance of Well-Being on Resiliency of Filipino Adults During the COVID-19 Enhanced Community Quarantine: A Necessary Condition Analysis

Conclusions about well-being being necessary but not sufficient for resilience may be unwarranted

Contact Info

Product

Resources

About