Culturally Responsive Assessments of Spatial Analytical Skills and Abilities: Development, Field Testing, and Implementation

Abstract

A persistent problem in education is underrepresentation of certain cultural and linguistic groups such as American Indian, African American, and Hispanic, in special programs for exceptionally talented students, especially in science, technology, engineering, and mathematics (STEM). The spatial analytical task, a performance-based assessment with demonstrated reliability and validity as an instrument to identify exceptionally talented students, was included with new instruments created in the Cultivating Diverse Talent in STEM (CDTIS) project. A continuum of problems, including closed, semi-open, and open-ended, was an important component of the measures, enabling the assessment of creative problem solving as well as assessment of skills such as seeing how things fit together visually and in space; through mental images, on paper, and in physical objects or forms. The spatial analytical assessment was implemented as part of a battery of instruments to identify students to participate in a special internship program. Ratings of students on the spatial analytical assessment who were identified for the internship program using the new assessments were higher than ratings for students identified using conventional methods, which demonstrates that the assessment will be a useful tool for selecting students from diverse cultural groups. The assessment has multiple purposes in addition to its use as an identification tool: evaluation of special programs, planning differentiated instruction, and as a pre- and post-measure of student gains. The test–retest reliability, and construct, concurrent, and predictive validity for secondary students need continued investigation in future studies.

Culturally Responsive Assessment of Life Science Skills and Abilities: Development, Field Testing, Implementation, and Results

“…The project was a collaboration between (a) faculty members at an R1 University in the Southwestern United States and (b) educators from four schools: two public schools, one charter school, and one Bureau of Indian Affairs grant school. Six assessments were included in the identification of students with exceptional talent in STEM: (a) math problem solving (Bahar & Maker, 2020), (b) spatial analytical performance assessment (Maker, 2020b), (c) physics concept maps (Maker & Zimmerman, 2020), (d) mechanical–technical performance assessment (Alfaiz et al, 2020), (e) life science concept maps (Maker & Zimmerman, 2020), and (f) the life science performance assessment. These six assessments were in three domains, math, physical sciences, and life sciences, and were measures of students’ domain-specific skills and abilities, creativity-relevant processes, and task motivation (Amabile, 2013).…”

Identifying Exceptional Talent in Science, Technology, Engineering, and Mathematics: Increasing Diversity and Assessing Creative Problem-Solving

“…Consistent with this perspective, Sak and Maker (2006), in a study of mathematical problem-solving using the Discovering Intellectual Strengths and Capabilities while Observing Varied Ethnic Responses (DISCOVER) assessment, concluded that in elementary-age students, two standard deviations above the mean of domain-specific knowledge was a threshold for a one standard deviation above the mean of creativity in math. The DISCOVER math assessment (Bahar & Maker, 2011, 2020Maker, 1993Maker, , 2005, created in 1992 from a perspective similar to that of Silvia (2015), includes solving known problems with known operations and correct answers, accuracy in the solving of other types of problems (domain-specific knowledge and skills), and solving problems with multiple correct answers by applying mathematical concepts and operations in many and novel ways (domain-specific and domain-general creative processes; e.g., write as many problems as you can with the answer of 10).…”

“…Based on field tests, instruments and observer instructions were revised again, and after field testing and additional revisions as needed, instruments were administered to the students in Grade 11 at the same partner schools. Development of each of these instruments and an in-depth description of their current form can be found in other publications (Alfaiz et al, 2020;Bahar & Maker, 2020;Maker, 2020;Zimmerman et al, 2020).…”

“…The core competencies, problem types, and tasks for each assessment are summarized in Table 2. Each assessment is described in depth in other publications (Alfaiz et al, 2020;Bahar & Maker, 2020;Maker, 2020;Zimmerman et al, 2020).…”

“…In most cases, all six assessments, which included life science performance-based assessment (Zimmerman et al, 2020), physical science (mechanical-technical) performance-based assessment (Alfaiz et al, 2020), spatial analytical performance-based assessment (Maker, 2020), concept maps , and math (Bahar & Maker, 2020), were administered over a 2-day period at each school. Students gathered in the library or another large room at tables for the performance-based assessments, one observer for each three to five students, and for the paper-and-pencil assessments, they gathered in a large room with one research team member administering the assessment and two to four others assisting as monitors.…”