In Pursuit of Validity: A comprehensive review of science attitude instruments 1935–2005

Abstract

The purpose of this review is to present a comprehensive evaluation of science attitude instruments based on published psychometric evidence. A multitude of instruments have been used through the years and some have been linked to career choice and school performance. Substantiating such associations is of paramount importance if researchers wish to influence educational policy. However, associations are reduced, or hard to discover, if instruments have weak psychometric properties. Several databases were searched for peer-reviewed articles that discussed the development and use of science attitude instruments. Instruments were grouped into the following categories: attitudes toward science, scientific attitudes, nature of science, scientific career interests, and other. A data abstraction and scoring rubric was used to summarize and evaluate 150 published articles that spanned 66 instruments. Most instruments had single study usage and showed an absence of psychometric evidence. This review demonstrated that there are few instruments available with the necessary psychometric data to merit recommendation. The review quantifies the current state of the research regarding the measurement of science attitude in students; the results should elicit further discussion and encourage more rigorous analyses of instruments. The findings may assist other researchers to select an instrument and alert them to its strengths and weaknesses. This review points the way forward for research in this field. Instruments already in existence should be used in repeat studies, and reliability and validity evidence should be collected and shared.

Primary teachers' attitudes toward science: A new theoretical framework

“…Many studies provide incomplete definitions (or no definition at all) for the construct of attitude, fail to explicate the components of attitude that they measured, or do not distinguish between attitudes toward science and other related concepts (e.g., opinions or motivation). It is therefore difficult to determine exactly what is being measured or investigated (Blalock et al, 2008). Furthermore, most researchers fail to justify their choices to measure certain dimensions or objects of attitude.…”

“…In addition, most researchers failed to conduct pilot testing, validation, and evaluation of their measurement instrument according to current psychometric standards (Blalock et al, 2008;Coulson, 1992;Gardner, 1995;Reid, 2006). These methodological problems have at least two important consequences.…”

“…In addition to (and probably also because of) the poor theoretical definition of what constitutes primary teachers' attitudes toward science, many researchers have used poorly designed measurement instruments or inadequate methods of analysis (for a review of these issues, see Blalock et al, 2008;Coulson, 1992;Gardner, 1995;Reid, 2006). In addition, most researchers failed to conduct pilot testing, validation, and evaluation of their measurement instrument according to current psychometric standards (Blalock et al, 2008;Coulson, 1992;Gardner, 1995;Reid, 2006).…”

“…However, we do discuss several methodological problems when they affected the evaluation of the concepts that we derived. For extensive reviews of methodological issues related to the study of attitudes toward science, we refer to Blalock et al (2008) and Osborne et al (2003). In the remainder of this section, we introduce the general psychological concept of attitude and provide some important considerations that serve to clarify the measurable objects of primary teachers' attitudes toward science.…”

Should we embed careers education in STEM lessons?

“…Mindful of criticisms (e.g. Blalock et al, 2008) that science attitude surveys typically possess weak psychometric properties, a high proportion of the items for the student questionnaire were taken from well-validated constructs in the psychology literature that it seemed reasonable to hypothesise might be related to participation / intention to participate in mathematics and/or physics post-16 (Reiss et al, 2011). A total of approximately 23,000 students completed these questionnaires in either year 8 or 10 and approximately 7000 of these students completed them two years later.…”