Validating Chemistry Faculty Members’ Self-Reported Familiarity with Assessment Terminology

Abstract

With the increasing emphasis placed upon chemistry instructors and departments to assess and evaluate their courses and curricula, understanding the structure of chemistry faculty members' knowledge and use of assessment terms and concepts can shed light on potential areas for targeted professional development. Survey research that might accomplish this objective often relies on self-reported responses from the target audience, and such information is sometimes difficult to assess in terms of validity. As an example of an internal mechanism to help establish validity, it is possible to include an "internal standard" item early in the survey. For the sake of understanding faculty members' familiarity with assessment terminology, an item that asked participants to identify analogous pairs of terms comparing assessment measures (assessment validity and assessment reliability) to laboratory measures (accuracy and precision) served this purpose. Using ordered logistic regression, participants who answered the analogy question completely correctly were more likely to report higher levels of familiarity with the assessment terms. Because the self-reported data appears to be valid, these data can be further used in subsequent analyses in order to determine the general familiarity trends among chemistry faculty regarding assessment terminology.

Exploring STEM postsecondary instructors’ accounts of instructional planning and revisions

“…National reports have advocated a rethinking of the prominence of summative assessment, emphasizing the benefits of formative assessment within the college STEM classroom (National Research Council, 2003a). Survey-based studies in chemistry indicate that instructors have at least a mid-level understanding of the difference between these two types of assessment (Emenike, Raker, & Holme, 2013;Raker & Holme, 2014).…”

The Faculty Self-Reported Assessment Survey (FRAS): Differentiating Faculty Knowledge and Experience in Assessment

“…The first of these sections is a self-report of vocabulary items. This particular list of vocabulary items was based on prior work by the ACS and reported by Emenike et al . (2013b) .…”

“…The underpinning assumption of asking about familiarity with assessment vocabulary is that this reflects levels of exposure and ability in terms of assessment. The validity of this assumption was partially evaluated by Emenike et al . (2013b) by regressing vocabulary item response with an assessment expertise analogy task.…”

Designing Assessment Tools To Measure Students’ Conceptual Knowledge of Chemistry

“…In some ways, validity and reliability are akin to the chemistry constructs of accuracy and precision, respectively (23,24).…”