On Evaluating Human Problem Solving of Computationally Hard Problems

Abstract

This article is concerned with how computer science, and more exactly computational complexity theory, can inform cognitive science. In particular, we suggest factors to be taken into account when investigating how people deal with computational hardness. This discussion will address the two upper levels of Marr's Level Theory: the computational level and the algorithmic level. Our reasons for believing that humans indeed deal with hard cognitive functions are threefold: (1) Several computationally hard functions are suggested in the literature, e.g., in the areas of visual search, visual perception and analogical reasoning, linguistic processing, and decision making. This article gives a brief introduction to some theories and foundations of complexity theory and motivates the use of computationally hard problems in human problem solving with a short survey of known results of human performance, a review of some computationally hard games and puzzles, and the connection between complexity theory and models of cognitive functions. We aim to illuminate the role that computer science, in particular complexity theory, can play in the study of human problem solving. Theoretical computer science can provide a wealth of interesting problems for human study, but it can also help to provide deep insight into these problems. In particular, we discuss the role that computer science can play when choosing computational problems for study and designing experiments to investigate human performance. Finally, we enumerate issues and pitfalls that can arise when choosing computationally hard problems as the subject of study, in turn motivating some interesting potential future lines of study. The pitfalls addressed include: choice of presentation and representation of problem instances, evaluation of problem comprehension, and the role of cognitive support in experiments. Our goal is not to exhaustively list all the ways in which these choices may impact experimental studies, but rather to provide a few simple examples in order to highlight possible pitfalls. Keywords problem solving, computational complexity, intractability, algorithmic level theories, cognitive functions 1 University of Victoria. Please direct correspondence to scarruth@uvic.ca.

Conceptual Transformation and Cognitive Processes in Origami Paper Folding

“…(Chi, Glaser, & Rees, 1982). This is reflected in the relevant literature such as Newell and Simon (1972), and more recently in many contributions in the Journal of Problem Solving-compare discussions in Carruthers andStege (2013) andFischer, Greiff, andFunke (2012), and in introductory reviews such as Anderson (2004), which focus on the complex high-level operations that need to be mentally organised, based on the range of possible actions and problem states. It is in this area that think-aloud protocols as data sources have been most successful (Ericsson & Simon, 1984).…”

“…Accordingly, much of the problem solving literature addresses how people identify problems and operators to solve them, and how these operators are ordered into a sequence of actions so as to reach a suitable solution, mediated by expertise (Chi, Glaser, & Rees, 1982). This is reflected in the relevant literature such as Newell and Simon (1972), and more recently in many contributions in the Journal of Problem Solving-compare discussions in Carruthers and Stege (2013) and Fischer, Greiff, and Funke (2012), and in introductory reviews such as Anderson (2004), which focus on the complex high-level operations that need to be mentally organised, based on the range of possible actions and problem states.…”

An experiential program on the foundations of quantum mechanics for final-year high-school students

“…Research in different fields highlighted the role of games in favouring the connection between abstraction and formalization. Games designed with a purpose (GWAPs) can be powerful tools to boost research and science education [29][30][31][32][37][38][39][40][41][42][43], in particular favouring the use of intuition to quickly solve very complex computational problems [44], and reinforcing motivations and voluntary participation, ensuring…”

Cognitive and Computational Complexity: Considerations from Mathematical Problem Solving

“…The design of pertinent experiments is an enormous challenge and even with ideal circumstances for testing we cannot be entirely sure that we have accurately identified the algorithmic procedures. While that does not mean that we cannot gather reliable data at all, it does bring in the kind of inexactness that is a bad fit with the exact notions involved in the study of computational complexity (see, e.g., Carruthers and Stege 2013).…”