Measuring Collaborative Problem Solving in Low-Stakes Tasks

Assessment Gaze, Refraction, and Blur

“…Although performance assessments and portfolios were much in vogue in the late 1980s and 1990s, public questions about appropriateness of the content of tasks, their technical quality, and the time and costs involved led to a reduction in the popularity of its use in large-scale assessment, except for essays and fairly routine problem solving. At the beginning of the 21st century, however, new predictions of needed skills and knowledge revived performance-based tasks, including collaborative problem solving (O’Neil & Chuang, 2008), teamwork (O’Neil, Chung, & Brown, 1997), and transfer or application of knowledge to new settings or problems (Cooper & Sweller, 1987; Bransford, Brown, & Cocking, 2000). One tactic to support the continued use of performance tests was to place them in classrooms where they were to function less as formal tests and more as integral parts of instruction or embedded in formative assessment (Black & Wiliam, 2009; Phelan, Choi, Vendlinski, Baker, & Herman, 2011).…”

“…Great variety of computer-based assessment tasks have developed over the past 20 years, including domain-independent methods to measure conceptual understanding through mapping (e.g., E. L. Baker, Niemi, & Herl, 1994; Herl, O’Neil, Chung, & Schacter, 1999), problem solving (Vendlinski & Stevens, 2002), science inquiry (e.g., Bennett, Persky, Weiss, & Jenkins, 2007; Quellmalz et al, 2013), teamwork and collaborative problem solving (e.g., Chung, O’Neil, & Herl, 1999; O’Neil & Chuang, 2008; O’Neil, Chung, & Brown, 1997), interpersonal skills (e.g., O’Neil, Allred, & Dennis, 1997), and domain-dependent methods, including measurement of math reasoning (e.g., Bennett, Morley, & Quardt, 2000), Web searching (e.g., Klein, Yarnall, & Glaubke, 2002; Schacter, Herl, Chung, Dennis, & O’Neil, 1999), specific design skills (e.g., Bejar, 1991; Chung & Baker, 2003; Katz & James, 1998), and medical decision making (e.g., Clauser, 2000, Margolis & Clauser, 2006). Automated essay scoring has been under way for a number of years and is included because of its scoring methods and cost savings in comparison with other current methods (e.g., Burstein, Tetreault, & Madnani, 2013; Landauer, Laham, & Foltz, 2003; Page, 1966; Page & Petersen, 1995; Shermis, 2014).…”

Computer-based Assessment of Collaborative Problem Solving: Exploring the Feasibility of Human-to-Agent Approach

“…Finally, the study implemented a certain set of measures and techniques to assess CPS. Various research methodologies and measures developed in previous studies of CPS, collaborative learning, and teamwork processes potentially can be adapted to CPS assessment (e.g., Biswas et al 2005;Hsieh and O'Neil 2002;O'Neil and Chuang 2008;Rosen and Rimor 2012;Weinberger and Fischer 2006).…”

“…Students emerging from schools into the workforce and public life will be expected to have CPS skills as well as the ability to perform that collaboration in various group compositions and environments (Griffin et al 2012;OECD 2013;O'Neil and Chuang 2008;Rosen and Rimor 2012). Recent curriculum and instruction reforms have focused to a greater extent on teaching and learning CPS (National Research Council 2011; U.S. Department of Education 2010).…”