Commentary: False-Positive Effect in the Radin Double-Slit Experiment on Observer Consciousness as Determined With the Advanced Meta-Experimental Protocol

Response: Commentary: False-Positive Effect in the Radin Double-Slit Experiment on Observer Consciousness as Determined With the Advanced Meta-Experimental Protocol

“…The research record for this funder-commissioned study shows clearly, however, that the used analytic method-a chi-square test-was not pre-specified in advance. That is, regarding the chi-square test analysis, the research record contradicts the recent claim by Radin et al (2020) that "the analytic methods in the experiment were established before to prevent p-hacking." Thus, regrettably, this claim by Radin et al (2020) amounts to an example of the malpractice of HARKing.…”

“…That is, regarding the chi-square test analysis, the research record contradicts the recent claim by Radin et al (2020) that "the analytic methods in the experiment were established before to prevent p-hacking." Thus, regrettably, this claim by Radin et al (2020) amounts to an example of the malpractice of HARKing. Briefly, the simple reason why this chi-square test was present in the Matlab-script, which Radin et al (2020) explained that they had "examined, " is the following: It was a post-hoc modified version of the script-sent to the authors by Radin himself-at an earlier point in time, but years after the original pre-specified analysis had already been completed.…”

“…First, Radin et al (2020) claimed that Walleczek and von Stillfried (2019) failed to report a "truepositive outcome" for this experiment. The research record for this funder-commissioned study shows clearly, however, that the used analytic method-a chi-square test-was not pre-specified in advance.…”

“…Thus, regrettably, this claim by Radin et al (2020) amounts to an example of the malpractice of HARKing. Briefly, the simple reason why this chi-square test was present in the Matlab-script, which Radin et al (2020) explained that they had "examined, " is the following: It was a post-hoc modified version of the script-sent to the authors by Radin himself-at an earlier point in time, but years after the original pre-specified analysis had already been completed. Crucially, that version of the script contained, in addition to the planned statistical analyses, Radin's post-hoc analysis based on the (non-directional) chi-square test.…”

“…the experiment involved eight comparisons performed on non-overlapping data partitioned from a single dataset." For explanation, Radin et al (2020) here falsely suggest that the eight different hypotheses merely posited the random occurrence of (arbitrary) effects for "a single dataset"; however, in truth, a different specific prediction was tested with each of eight dedicated (single) data sets (Walleczek and von Stillfried, 2019). Therefore, statistical corrections for non-predictive, i.e., random, multiple testing would thus violate the AMPbased test strategy which implemented the planned outcome predictions as pre-specified by Radin in 2011: True-positive effects were predicted for only two (12.5%) of the 16 possible measurement outcomes of the eight single-test categories.…”