One of the key ideas of evidence-centered assessment design (ECD) is that task features can be deliberately manipulated to change the psychometric properties of items. ECD identifies a number of roles that task-feature variables can play, including determining the focus of evidence, guiding form creation, determining item difficulty and discrimination, characterizing proficiency, and producing task variants. Assessment developers can use these task features to manipulate the psychometric properties of both conventional assessment formats and complex tasks embedded in simulations and games.Simulations and games tasks present additional challenges: even defining what corresponds to an item can be difficult. Often task features are determined by game or simulation logic rather than psychometric design. Despite these difficulties, the roles for task features identified in ECD are useful for analyzing the psychometric properties of embedded assessments in simulations and games. This article compares a conventional test of mathematics-problem-solving ability using word problems to an assessment of conceptual physics, creativity, and conscientiousness embedded in the game Newton’s Playground, describing how the task roles play out in each setting and how they can be used to manipulate the evidence provided by an assessment.

No abstract

The Tangram Help/Hurt Task (THHT) allows participants to help another participant win a prize (by assigning them easy tangrams), to hurt another participant by preventing them from winning the prize (by assigning them difficult tangrams), or to do neither (by assigning them medium tangrams) in offline or online studies. Consistent with calls for continued evidence supporting psychological measurement, we conducted a meta-analytic review of the THHT that included 52 independent studies involving 11,060 participants. THHT scores were associated with helping and hurting outcomes in theoretically predicted ways. Results showed that THHT scores were not only associated with short-term (experimental manipulations, state measures) and long-term (trait measures) helping and hurting outcomes, but also with helping and harming intentions. We discuss the strengths and limitations of the THHT relative to other laboratory measures of prosocial behavior and aggression, discuss unanswered questions about the task, and offer suggestions for the best use of the task.

No abstract