Development and Psychometric Testing of the Rate and Accumulation Concept Inventory

Abstract

Background Because a fundamental understanding of rate and accumulation principles is important for engineering students across all subdisciplines, a method is needed to assess their understanding of these principles. Purpose/Hypothesis This article discusses the development of the Rate and Accumulation Concept Inventory (RACI) instrument and provides an analysis of its validity and reliability. The RACI is designed to test students’ (1) overall mastery of rate and accumulation concepts and (2) mastery of specific rate and accumulation concepts (e.g., mathematics, mass flow, and heat flow). Design/Methods Exploratory findings of students’ conceptual understanding prompted the development of a preliminary RACI instrument. Two pilot studies were conducted, with adjustments made to the instrument between each. Data from the second pilot were used to assess evidence of validity and reliability through classical test theory, item response theory, and structural equation modeling. Results Validity and reliability evidence indicates that the RACI can be used to measure students’ overall understanding of the concepts identified, though several items require revision or removal. Issues of potential construct underrepresentation were found in two of the conceptual categories. Conclusions The evidence of reliability and validity indicates the appropriateness of the RACI for assessing engineering students’ overall understanding of rate and accumulation principles. Additional research is needed to further enhance the validity and reliability of the RACI as a diagnostic tool, particularly in measuring students’ understanding of rate and accumulation concepts in particular physical contexts.