Culturally Responsive Assessment of Physical Science Skills and Abilities: Development, Field Testing, Implementation, and Results

Abstract

During the Cultivating Diverse Talent in STEM (CDTIS) Project, a team of scientists, teachers, and a researcher developed a performance-based assessment of high school students’ creative problem-solving skills and ability to apply physical science principles in practical ways. It was one of six measures to identify exceptionally talented students. Students identified using conventional methods (M1), with an average grade point average (GPA) of 3.93, had an average rating of 2.95 on a 5-point scale on the mechanical–technical assessment. The M2 students, who were from schools with high percentages of Hispanic, American Indian, and low socioeconomic status (SES) students, had an average GPA of 3.07 and an average rating of 3.27, demonstrating that this assessment can be an important way to change the cultural and economic balance of students identified as exceptionally talented in Science, Technology, Engineering, and Mathematics (STEM). Other researchers are encouraged to examine the validity of the mechanical–technical assessment to identify exceptionally talented students in different groups.

Culturally Responsive Assessments of Spatial Analytical Skills and Abilities: Development, Field Testing, and Implementation

“…Overall results are provided in another publication (Maker, 2020). Descriptions of the new performance assessments used in the selection process for M2 students are provided in other publicatons: concept maps (Maker & Zimmerman, 2020), math (Bahar & Maker, 2020), life science performance assessment (Zimmerman et al, 2020), physical science performance assessment (Alfaiz et al, 2020). However, in this section the results of the spatial analytical and math assessments in each partner school are compared to demonstrate the utility of the assessment to identify exceptional talent in students from diverse cultural groups and low-income families (Table 2).…”

Identifying Exceptional Talent in Science, Technology, Engineering, and Mathematics: Increasing Diversity and Assessing Creative Problem-Solving

“…In their continuum (Table 1) (Table 2). For example, in the assessment of mechanical-technical abilities (closely related to physics and engineering), the closed problem was to make a gear box given a set of materials and a diagram, the semi-open problem was to make one of two vehicles based on a picture of the final product and a picture of the materials needed to make it, but with no instructions telling how to put the pieces together to make the vehicle, and the open-ended problem was to make a machine that moved with the remote and motors that was different from the vehicle and of the student's own creation (Alfaiz et al, 2020).…”

“…Based on field tests, instruments and observer instructions were revised again, and after field testing and additional revisions as needed, instruments were administered to the students in Grade 11 at the same partner schools. Development of each of these instruments and an in-depth description of their current form can be found in other publications (Alfaiz et al, 2020;Bahar & Maker, 2020;Maker, 2020;Zimmerman et al, 2020).…”

“…The core competencies, problem types, and tasks for each assessment are summarized in Table 2. Each assessment is described in depth in other publications (Alfaiz et al, 2020;Bahar & Maker, 2020;Maker, 2020;Zimmerman et al, 2020).…”

“…In most cases, all six assessments, which included life science performance-based assessment (Zimmerman et al, 2020), physical science (mechanical-technical) performance-based assessment (Alfaiz et al, 2020), spatial analytical performance-based assessment (Maker, 2020), concept maps , and math (Bahar & Maker, 2020), were administered over a 2-day period at each school. Students gathered in the library or another large room at tables for the performance-based assessments, one observer for each three to five students, and for the paper-and-pencil assessments, they gathered in a large room with one research team member administering the assessment and two to four others assisting as monitors.…”