Proving a negative? Methodological, statistical, and psychometric flaws in Ullmann et al. (2017) PTSD study

Abstract

Ullmann et al. recently published a pilot study in Translational Psychiatry in which they report failing to find a statistically significant reduction in either hair cortisol or hair cortisone levels in circumcised men as compared with genitally intact (noncircumcised) men. Based on such null findings, the authors purport to have "refuted the psycho-pathological long-term effects of circumcision" and that the lack of significant results, "add to the growing body of evidence in the literature that male circumcision is not likely psychologically traumatizing across the life-span." In addition, they claim that they have proven a "healthy functionality of the LHPA axis" in men subjected to circumcision during infancy or childhood. However, it is not possible to draw any such conclusions on the basis of a null finding, especially one derived from an underpowered study in which the trend in the data suggest, if anything, that an adequately powered study may have shown the opposite of what the authors claim. Relevance for patients:When combined with other weaknesses in study design, measurement, and interpretation, it becomes apparent that the authors' conclusions are not supported by their data.

Neonatal male circumcision is associated with altered adult socio-affective processing

“…Relatively little research has investigated pain-related responses to circumcision in infants ( Dixon et al., 1984 ; Fergusson et al., 2007 ; Gattari et al., 2013 ; Gunnar et al., 1981 , 1995 ; Marshall et al., 1980 , 1982 ; Mondzelewski et al., 2016 ; Page, 2004 ; Svoboda and Van Howe, 2013 ; Taddio et al., 1997a , 1997b ; Talbert et al., 1976 ; Williamson et al., 1986 ; Williamson and Williamson, 1983 ), and only a handful of studies have generated evidence regarding potential long-term effects lasting into adulthood ( Bauer and Kriebel, 2013 ; Bollinger and Van Howe, 2011 ; Frisch and Simonsen, 2015a ; Ullmann et al., 2017 ). The small number of studies that have investigated long-term effects have been argued to exhibit methodological flaws and limitations ( Boyle, 2017 ; Morris et al., 2012 ; Morris and Wiswell, 2015 ) leading to calls for further research in this area ( Bollinger and Van Howe, 2012 ; Frisch and Simonsen, 2015b ).…”

“…It is also contentious whether early-circumcised males experience long-term alterations within the limbic-hypothalamic-pituitary-adrenocortical (LHPA) system; and whether potential stress in this regard is connected to developmental factors. The only study that, to our knowledge, examined this issue was statistically underpowered ( Boyle, 2017 ), using a sample of only 9 circumcised participants ( Ullmann et al., 2017 ). Nevertheless, despite being underpowered, the trend in the data were consistent with the hypothesized decrease in long-term activation of the LHPA axis among early-circumcised males, as assessed through both hair cortisol and cortisone levels ( Boyle, 2017 ; Ullmann et al., 2017 ).…”

“…The only study that, to our knowledge, examined this issue was statistically underpowered ( Boyle, 2017 ), using a sample of only 9 circumcised participants ( Ullmann et al., 2017 ). Nevertheless, despite being underpowered, the trend in the data were consistent with the hypothesized decrease in long-term activation of the LHPA axis among early-circumcised males, as assessed through both hair cortisol and cortisone levels ( Boyle, 2017 ; Ullmann et al., 2017 ). Finally, one recent study has found that neonatal male circumcision and prematurity were associated with Sudden Infant Death Syndrome (SIDS), providing support for the allostatic load hypothesis (i.e., the result of cumulative perinatal painful, stressful, or traumatic events) as an explanation for SIDS ( Elhaik, 2018 ).…”

Critical evaluation of arguments opposing male circumcision: A systematic review

“…An MC opponent commented that the study was underpowered. 89 A larger confirmation study would help address this.…”

“…Rather, the study found that an increase in the glucocorticoid levels indicated a healthy lifestyle and appropriate functioning, concluding that the study provided evidence that MC does not promote psychological trauma. An MC opponent commented that the study was underpowered . A larger confirmation study would help address this.…”