When Less Is Less: Solving Multiple Simple Problems Is Not Complex Problem Solving—A comment on Greiff et al. (2015)

Abstract

Abstract:In this commentary, we critically review the study of Greiff, Stadler, Sonnleitner, Wolff, and Martin, "Sometimes less is more: Comparing the validity of complex problem solving measures" (Intelligence, 2015, 50, 100-113). The main conclusion of Greiff et al. that the "multiple complex systems" (MCS) approach to measuring complex problem-solving ability possesses superior validity compared to classical microworld scenarios ("less is more") seems to be an overgeneralization based on inappropriate analysis and selective interpretation of results. In its original form, MCS is a useful tool for investigating specific aspects of problem solving within dynamic systems. However, its value as an instrument for the assessment of complex problem solving ability remains limited.

Impact of Cognitive Abilities and Prior Knowledge on Complex Problem Solving Performance – Empirical Results and a Plea for Ecologically Valid Microworlds

“…For the latter, the quality of one’s hypotheses, which is based on domain knowledge, is a necessary prerequisite for successfully exploring the system. In summary, the features of MCS measurements outlined here, along with further criticisms of this approach (e.g., Funke, 2014; Scherer, 2015; Schoppek and Fischer, 2015; Dörner and Funke, 2017; Funke et al, 2017; Kretzschmar, 2017), substantially narrow the validity of the MCS approach as an indicator of CPS.…”

“…Studies provide support for the psychometric quality, especially the reliability, of the MCS approach, although scale building and some statistics have been criticized (Funke et al, 2017; Kretzschmar, 2017). Only one study so far has attempted to compare MCS and CRS.…”

“…This approach has recently become prominent in educational psychology (e.g., Greiff et al, 2013b; Sonnleitner et al, 2013; Kretzschmar et al, 2014; OECD, 2014; Csapó and Molnár, 2017). In addition, this shift has led to the question of what are and are not complex problems, with some researchers questioning the relevance of MCS as a tool for CPS research and the validity of the conclusions drawn from them (e.g., Funke, 2014; Dörner and Funke, 2017; Funke et al, 2017; Kretzschmar, 2017). …”

Complex Problem Solving: What It Is and What It Is Not

“…Numerous articles on the subject have been published in recent years, documenting the increasing research activity relating to this field. In the following collection of papers we list only those published in 2010 and later: theoretical papers (Blech and Funke, 2010; Funke, 2010; Knauff and Wolf, 2010; Leutner et al, 2012; Selten et al, 2012; Wüstenberg et al, 2012; Greiff et al, 2013b; Fischer and Neubert, 2015; Schoppek and Fischer, 2015), papers about measurement issues (Danner et al, 2011a; Greiff et al, 2012, 2015a; Alison et al, 2013; Gobert et al, 2015; Greiff and Fischer, 2013; Herde et al, 2016; Stadler et al, 2016), papers about applications (Fischer and Neubert, 2015; Ederer et al, 2016; Tremblay et al, 2017), papers about differential effects (Barth and Funke, 2010; Danner et al, 2011b; Beckmann and Goode, 2014; Greiff and Neubert, 2014; Scherer et al, 2015; Meißner et al, 2016; Wüstenberg et al, 2016), one paper about developmental effects (Frischkorn et al, 2014), one paper with a neuroscience background (Osman, 2012) 1 , papers about cultural differences (Güss and Dörner, 2011; Sonnleitner et al, 2014; Güss et al, 2015), papers about validity issues (Goode and Beckmann, 2010; Greiff et al, 2013c; Schweizer et al, 2013; Mainert et al, 2015; Funke et al, 2017; Greiff et al, 2017, 2015b; Kretzschmar et al, 2016; Kretzschmar, 2017), review papers and meta-analyses (Osman, 2010; Stadler et al, 2015), and finally books (Qudrat-Ullah, 2015; Csapó and Funke, 2017b) and book chapters (Funke, 2012; Hotaling et al, 2015; Funke and Greiff, 2017; Greiff and Funke, 2017; Csapó and Funke, 2017a; Fischer et al, 2017; Molnàr et al, 2017; Tobinski and Fritz, 2017; Viehrig et al, 2017). In addition, a new “Journal of Dynamic Decision Making” (JDDM) has been launched (Fischer et al, 2015, 2016) to give the field an open-access outlet for research and discussion.…”

Competencies for Complexity: Problem Solving in the Twenty-First Century

“…In this chapter, we present a view of problem solving competency as a bundle of skills, knowledge and abilities that are required to deal effectively with complex and dynamic non-routine situations in different domains (Fischer and Neubert 2015). We consider this change of perspective important to move away from the relatively narrow conception of problem solving ability based on a conventional psychometric perspective which has become prominent in recent years (Funke 2014a, b;Funke et al 2017;Schoppek and Fischer 2015). The components of this competency do not necessarily need to be correlated, which is often implied in the psychometric view on problem solving.…”