Initial data analysis: A new technology not yet ready to use

Abstract

In the ongoing discussion on improving quality and reproducibility of research, the data pipeline had been identified as one crucial element. Everything happening with the data already prior to the final analysis may have a highly relevant impact on the final analysis itself and its interpretation. Consequently, there is an increased focus on these processes. Sound practices for initial data analyses (IDA) have been described and the rigorous use of IDA has been advocated. However, IDA can only have a positive impact on research quality if findings from a systematic data screening lead to reasonable changes in the final analysis or its interpretation. In this communication some general challenges in finding adequate responses to findings from data screening are identified. They reflect a lack of knowledge and consensus. Consequently, rigorous future use of IDA can only be advocated if guidance for responding to data screening findings is further developed. With the knowledge present today, IDA may do more harm than good.