Missing the Wood for the Wrong Trees: On the Difficulty of Defining the Complexity of Complex Problem Solving Scenarios

Abstract

Abstract:In this paper we discuss how the lack of a common framework in Complex Problem Solving (CPS) creates a major hindrance to a productive integration of findings and insights gained in its 40+-year history of research. We propose a framework that anchors complexity within the tri-dimensional variable space of Person, Task and Situation. Complexity is determined by the number of information cues that need to be processed in parallel. What constitutes an information cue is dependent on the kind of task, the system or CPS scenario used and the task environment (i.e., situation) in which the task is performed. Difficulty is conceptualised as a person's subjective reflection of complexity. Using an existing data set of N = 294 university students' problem solving performances, we test the assumption derived from this framework that particular system features such as numbers of variables (NoV) or numbers of relationships (NoR) are inappropriate indicators of complexity. We do so by contrasting control performance across four systems that differ in these attributes. Results suggest that for controlling systems (task) with semantically neutral embedment (situation), the maximum number of dependencies any of the output variables has is a promising indicator of this task's complexity.

Beyond Psychometrics: The Difference between Difficult Problem Solving and Complex Problem Solving

“…In the latter case, problem solvers would have already possessed knowledge that they were expected to acquire during the subsequent exploration phase. Both the average hboxtextitP r value of around zero and the fact that expectations regarding the existence of relationships are equally distributed across the 12 possible variable links replicates what was found in earlier studies contrasting CPS scenarios with high and low semanticity ( Beckmann, 1994 ; Beckmann and Goode, 2014 , 2017 ). A counterfactual causal structure can therefore be ruled out as an alternative explanation for a potential semantic effect.…”

“…One symptom of a predominantly psychometric view on CPS is the lack of a distinction between complexity and difficulty ( Beckmann and Goode, 2017 ). Difficulty is a psychometric concept with rather limited explanatory value.…”

“…As has been discussed previously ( Beckmann, 2010 ; Birney et al, 2016 ; Beckmann and Goode, 2017 ), psychological research takes place in the three dimensional space of Person, Task, and Situation variables. The definition of task has two sub-facets, the task qua task and the task as behavior requirement ( McGrath and Altman, 1966 ; Hackman, 1969 ; Wood, 1986 ).…”

“…Subsequent interventions should then focus on the effects of each input variable on any of the output variables in isolation, i.e., changing only one input at a time. Only such “Vary-One-or-None-At-a-Time” heuristic (VONAT, see Beckmann and Goode, 2014 ; p. 279; Beckmann and Goode, 2017 ) creates informative system state transitions that allow problem solvers to derive knowledge regarding the causal structure of the system. In contrast, changing more than one variable at a time or to miss the zero change intervention creates what de Jong and van Joolingen (1998 , p. 185) describe as “inconclusive experiments,” which impedes successful knowledge acquisition.…”

Moderators of learning and performance trajectories in microworld simulations: Too soon to give up on intellect!?

“…In addition, our study goes beyond the typical methodology found in traditional individual differences research by sampling experienced mid-level senior managers. This combination provides an effective methodological paradigm for investigating real-world learning and performance (Beckmann & Goode, 2017;Goode & Beckmann, 2010), while allowing for experimental manipulations that are necessary to isolate the moderating effects of other variables. Thus, rather than modelling performance using only correlational data, our approach is to consider within-subject variability in decision making in the context of training using a repeated-measures design that considers learning and performance processes that co-occur and evolve over time.…”