Toolkit for detecting misused epidemiological methods

Abstract

Background Critical knowledge of what we know about health and disease, risk factors, causation, prevention, and treatment, derives from epidemiology. Unfortunately, its methods and language can be misused and improperly applied. A repertoire of methods, techniques, arguments, and tactics are used by some people to manipulate science, usually in the service of powerful interests, and particularly those with a financial stake related to toxic agents. Such interests work to foment uncertainty, cast doubt, and mislead decision makers by seeding confusion about cause-and-effect relating to population health. We have compiled a toolkit of the methods used by those whose interests are not aligned with the public health sciences. Professional epidemiologists, as well as those who rely on their work, will thereby be more readily equipped to detect bias and flaws resulting from financial conflict-of-interest, improper study design, data collection, analysis, or interpretation, bringing greater clarity—not only to the advancement of knowledge, but, more immediately, to policy debates. Methods The summary of techniques used to manipulate epidemiological findings, compiled as part of the 2020 Position Statement of the International Network for Epidemiology in Policy (INEP) entitled Conflict-of-Interest and Disclosure in Epidemiology, has been expanded and further elucidated in this commentary. Results Some level of uncertainty is inherent in science. However, corrupted and incomplete literature contributes to confuse, foment further uncertainty, and cast doubt about the evidence under consideration. Confusion delays scientific advancement and leads to the inability of policymakers to make changes that, if enacted, would—supported by the body of valid evidence—protect, maintain, and improve public health. An accessible toolkit is provided that brings attention to the misuse of the methods of epidemiology. Its usefulness is as a compendium of what those trained in epidemiology, as well as those reviewing epidemiological studies, should identify methodologically when assessing the transparency and validity of any epidemiological inquiry, evaluation, or argument. The problems resulting from financial conflicting interests and the misuse of scientific methods, in conjunction with the strategies that can be used to safeguard public health against them, apply not only to epidemiologists, but also to other public health professionals. Conclusions This novel toolkit is for use in protecting the public. It is provided to assist public health professionals as gatekeepers of their respective specialty and subspecialty disciplines whose mission includes protecting, maintaining, and improving the public’s health. It is intended to serve our roles as educators, reviewers, and researchers.

Demonstrating the undermining of science and health policy after the Fukushima nuclear accident by applying the Toolkit for detecting misused epidemiological methods

“…The failure to include this quotation in the SHAMISEN review paper [5] demonstrates the misuse of epidemiology in relation to items A15 (Suppressing data), A17 (Biased reporting), and C3 (Failing to generalize health risks) of the Toolkit [4].…”

“…The papers by Hayashida et al [18,19] cited in the SHAMISEN review paper [5] exhibit the misuse of epidemiology in relation to items A2 (Ignoring Type II errors) and A3 (Inappropriately interpreting the statistical analysis or results) of the Toolkit [4].…”

“…The statement also listed dozens of examples of ways to manage COI that include identification, avoidance, disclosure, and recusal [3]. To recognize distorted and misapplied epidemiological science, techniques used to manipulate epidemiological findings, summarized as part of the INEP Statement in 2020, were expanded and further elucidated in the article entitled, "Toolkit for detecting misused epidemiological methods" (hereinafter, the Toolkit) [4]. This Toolkit consists of 33 items that directly relate to the inappropriate application (or, misuse) of the epidemiological method.…”

“…This Toolkit consists of 33 items that directly relate to the inappropriate application (or, misuse) of the epidemiological method. The 33 items are organized in three categories: A) epidemiology-specific methods/techniques used to foment uncertainty and cast doubt about cause-and-effect (18 items); B) arguments used to delay action, maintain the status quo, and create division among scientists (8 items); and C) tactics invoked to misdirect policy priorities through influence (7 items) [4]. Each of these items is listed in Additional file 1.…”

“…Had the abovementioned review paper elaborated on the Chernobyl experience and the status of ultrasoundbased screening for thyroid cancer in Fukushima Prefecture using sufficient and correct information, readers of the SHAMISEN review paper would have come to a completely different conclusion. The purpose of our review is to provide information that the SHAMISEN review paper [5] failed to convey, to point out misinformation in the article using the Toolkit by Soskolne et al [4], and to help readers and policy-makers reach decisions based on factual scientific evidence.…”