Revealing Student Thinking about Experimental Design and the Roles of Control Experiments

Abstract

Well-designed "controls" distinguish experimental from non-experimental studies. Surprisingly, we found that a high percentage of students had difficulty identifying control experiments even after completing three university-level laboratory courses. To address this issue, we designed and ran a revised cell biology lab course in which students participated in weekly "experimental control exercises." To measure student understanding of control experiments, we developed a set of assessment questions; these were given to students prior to and following completion of either a standard cell biology lab course or the revised cell biology lab course. Not unexpectedly, the results indicate that the revised course led to greater improvements in students' ability to identify and explain the purpose of control experiments. Based on these observations, we recommend that explicit and detailed discussions designed to identify the design and purpose behind control experiments become a standard component of all laboratory courses.

CREATE Cornerstone: Introduction to Scientific Thinking, a New Course for STEM-Interested Freshmen, Demystifies Scientific Thinking through Analysis of Scientific Literature

“…At the same time, particularly for biology, the explosion of fundamental content makes it impossible for faculty to cover, let alone teach, “basic” material even to the same depth it was covered in the introductory courses of their own undergraduate years (Hoskins and Stevens, 2009). In addition, despite having encountered them in multiple courses, students may fail to retain key concepts (e.g., function of control experiments; see Shi et al ., 2011). Our compromise was to base a freshman course on particular examples of scientific literature, choosing topics in a limited range of content areas and focusing in-depth on scientific thinking and data analysis.…”

Using a discussion about scientific controversy to teach central concepts in experimental design

“…Students quickly identify the lack of random sampling or controls in the Wakefield study (Wakefield et al ). This is a positive outcome because previous work has shown that students struggle with designing experiments with appropriate controls (Shi et al ). In this exercise, students are also usually able to understand and explain that the lack of random sampling prevents cause and effect to be inferred from the data.…”

“…‘Experimental design’ is frequently emphasised as the most important aspect of a good study (Festing 2003), but the meaning of the phrase is not clearly articulated to students and is rarely addressed explicitly in learning activities (Chaplin ; Hiebert ). As a result, students starting an undergraduate science degree can have a very poor understanding of fundamental requirements of reliable, robust and valid experiments (Dasgupta et al ; Deane et al ; Festing ; Hiebert ; Pollack ; Shi et al ). However, in science subjects, the need for good experimental design is a threshold concept, one which learners need to master before they can make progress in their discipline (Meyer and Land ; ).…”

“…On examination of this point, many of them are taken aback that no controls were included in the original study. Crucially, having read the Offit and Coffin () paper, it is clear that students can identify what appropriate controls would be in this case (Hiebert ; Shi et al ). By specifically examining the need for controls, this exercise makes students more likely to identify and evaluate the controls used in future reading and include appropriate controls in experiments they design.…”

Online, Interactive Tutorials Provide Effective Instruction in Science Process Skills

“…These large-scale changes require significant investment of the instructor’s time and re-allocation of class time. Other attempts at emphasizing science process skills have focused on their integration into laboratory and research experiences, an option not available to all courses and instructors (DebBurman, 2002; Shi et al ., 2011; Brownell et al ., 2015; Woodham et al, 2016). Short of these large changes, could smaller scale incorporation of science process skills instruction prove useful to student learning?…”