Using a discussion about scientific controversy to teach central concepts in experimental design

Abstract

Students may need explicit training in informal statistical reasoning in order to design experiments or use formal statistical tests effectively. By using scientific scandals and media misinterpretation, we can explore the need for good experimental design in an informal way. This article describes the use of a paper that reviews the measles mumps rubella vaccine and autism controversy in the UK to illustrate a number of threshold concepts underlying good study design and interpretation of scientific evidence. These include the necessity of sufficient sample size, representative and random sampling, appropriate controls and inferring causation.

An Active Learning Intervention Based on Evaluating Alternative Hypotheses Increases Scientific Literacy of Controlled Experiments in Introductory Biology

“…Hypothesis 1 is that students would have difficulty identifying that confounding factors were unaccounted for when faced with a flawed scientific scenario and there would not be improvement throughout the course. The main evidence underlying the expected outcome of hypothesis 1 is the finding of multiple previous studies in which student understanding of controlled experiments did not improve after exposure to controlled experiments in undergraduate courses ( 6 – 10 ). After establishing the wide prevalence of this problem, we taught an introductory biology course that used (i) the framework centered on thinking about alternative hypotheses and confounding factors when determining cause-and-effect relationships together and (ii) multiple opportunities to practice application using case studies fully integrated with course content in an introductory biology course.…”

“…Shi and colleagues found that undergraduate students could not adequately explain experimental controls even after completing three laboratory courses ( 6 ). Other studies confirmed that exposure to controlled experiments is not sufficient to lead to a depth of understanding of controls ( 7 – 10 ).…”

Discovering experimental design: An interactive teaching exercise using Fisher's tea‐tasting experiment

“…19] and calculates that, if true, correctly classifying all eight cups would occur with probability 1=C 8 4 = 1/70 (see Data S1 for explanation). Correctly classifying three of the four MF, and hence also three of the four TF, would occur with probability C 4 3 Â C 4 1 À Á =C 8 4 = 16/70, which he states "could not be judged as statistically significant evidence of a real sensory discrimination" [13,p. 17].…”

“…Previous work highlights using accessible examples [17], such as those that relate to controversies in science [4]. The choice must be carefully tailored to the requirements of the course and the abilities of the students taking it.…”

Fostering comprehension of risk and causation through media case studies

“…Additionally, some articles from Teaching Statistics lend themselves to a similar analysis. Nordmoe's () piece on cancer in relation to cellphone usage and Bennett's () article on vaccines in relation to autism both provide timely case studies for instructors and students.…”