Development of the Exams Data Analysis Spreadsheet as a Tool To Help Instructors Conduct Customizable Analyses of Student ACS Exam Data

Abstract

The American Chemical Society Examinations Institute (ACS-EI) has recently developed the Exams Data Analysis Spread (EDAS) as a tool to help instructors conduct customizable analyses of their student data from ACS exams. The EDAS calculations allow instructors to analyze their students' performances both at the total score and individual item levels, while also providing national normative results that can be used for comparison. Additionally, instructors can analyze results based on subsets of items of their choosing or items based on the "big ideas" from the Anchoring Concepts Content Map (ACCM). In order to evaluate the utility and usability of the EDAS for instructors, the EDAS went through trial testing with 10 chemistry instructors from across the country. The instructor feedback confirmed that the EDAS has multiple implications for the classroom and departmental assessment, but some additional revisions were needed to increase its usability. This feedback was also used to make a video user-guide that will help instructors through specific difficulties described during trial testing. Currently, an EDAS tool has been developed for the GC12F, GC10S, and GC13 exams. Disciplines Other Chemistry | Science and Mathematics Education CommentsReprinted (adapted) with permission from J. Chem. Educ., 2015, 92 (12) ABSTRACT: The American Chemical Society Examinations Institute (ACS-EI) has recently developed the Exams Data Analysis Spread (EDAS) as a tool to help instructors conduct customizable analyses of their student data from ACS exams. The EDAS calculations allow instructors to analyze their students' performances both at the total score and individual item levels, while also providing national normative results that can be used for comparison. Additionally, instructors can analyze results based on subsets of items of their choosing or items based on the "big ideas" from the Anchoring Concepts Content Map (ACCM). In order to evaluate the utility and usability of the EDAS for instructors, the EDAS went through trial testing with 10 chemistry instructors from across the country. The instructor feedback confirmed that the EDAS has multiple implications for the classroom and departmental assessment, but some additional revisions were needed to increase its usability. This feedback was also used to make a video user-guide that will help instructors through specific difficulties described during trial testing. Currently, an EDAS tool has been developed for the GC12F, GC10S, and GC13 exams.

Applying an Interspersed Online and Offline Teaching Strategy in Medicinal Chemistry to Cultivate Research Interest and Innovation Skills in Undergraduate Pharmacy Students

“…An Educause Review article noted, "Institutions of higher learning are moving their operations to the cloud at a rapid pace" [6]. The cloud provides powerful means to foster undergraduate learning and enhance the quality of assessment [7]. Although it is still in the early stage for college educators, teaching through the internet with immense resources, an interspersed online and o ine teaching strategy, and other cloud related forms of education will gradually become popular educational models.…”

Evaluation of Subset Norm Stability in ACS General Chemistry Exams

“…5−8 Additionally, a tool was developed by the ACS-EI to aid instructors as they conducted analyses of student performance on standardized, norm-referenced ACS General Chemistry Exams. 9,10 This tool, the Exams Data Analysis Spreadsheet (EDAS), allows instructors to perform unique analyses of performance data by allowing instructor selection of exam items to include in the analyses. The EDAS then generates new norms and class performance comparisons upon the basis of the subset of items the user has selected for analysis.…”

“…Therefore, the ACS-EI has made efforts to translate research endeavors into pragmatic approaches for classroom assessment. For example, development of the Anchoring Concepts Content Map (ACCM) for various chemistry subdisciplines stemmed from desired instructional interest to analyze students’ content knowledge and provide the opportunity for programmatic assessment. − Additionally, a tool was developed by the ACS-EI to aid instructors as they conducted analyses of student performance on standardized, norm-referenced ACS General Chemistry Exams. , This tool, the Exams Data Analysis Spreadsheet (EDAS), allows instructors to perform unique analyses of performance data by allowing instructor selection of exam items to include in the analyses. The EDAS then generates new norms and class performance comparisons upon the basis of the subset of items the user has selected for analysis.…”

Development of a Method for Imputation of Missing Data Using ACS Exams as a Prototype

“…These content maps have many possible uses, with the primary use at ACS Exams being the alignment of exam items; − these alignments are then used for reporting performance by a set of criteria or criterion referencing . Additionally, ACS Exams has also reported on a method and automated process to generate subscores on the basis of item removal . In order to report performance by alignment to the content areas of the ACCM or subscoring based on item removal, individual item responses are needed.…”