This study investigates the internal structure and construct validity of Complex Problem Solving (CPS), which is measured by a Multiple-Item-Approach. It is tested, if (a) three facets of CPS -rule identification (adequateness of strategies), rule knowledge (generated knowledge) and rule application (ability to control a system) -can be empirically distinguished, how (b) reasoning is related to these CPS-facets and if (c) CPS shows incremental validity in predicting school grade point average (GPA) beyond reasoning. N = 222 university students completed Micro-DYN, a computer-based CPS test and Ravens Advanced Progressive Matrices. Analysis including structural equation models showed that a 2-dimensionsal model of CPS including rule knowledge and rule application fitted the data best. Furthermore, reasoning predicted performance in rule application only indirectly through its influence on rule knowledge indicating that learning during system exploration is a prerequisite for controlling a system successfully. Finally, CPS explained variance in GPA even beyond reasoning, showing incremental validity of CPS. Thus, CPS measures important aspects of academic performance not assessed by reasoning and should be considered when predicting real life criteria such as GPA.

With the advent of computers in the experimental labs, dynamic systems have become a new tool for research on problem solving and decision making. A short review of this research is given and the main features of these systems (connectivity and dynamics) are illustrated. To allow systematic approaches to the influential variables in this area, two formal frameworks (linear structural equations and finite state automata) are presented. Besides the formal background, the article sets out how the task demands of system identification and system control can be realised in these environments, and how psychometrically acceptable dependent variables can be derived.

Word Count (without references): 5.463 Revised version 15.10.09

Innovative assessments of cross-curricular competencies such as complex problem solving (CPS) have currently received considerable attention in large-scale educational studies. This study investigated the nature of CPS by applying a state-of-the-art approach to assess CPS in high school. We analyzed whether two processes derived from cognitive psychology, knowledge acquisition and knowledge application, could be measured equally well across grades and how these processes differed between grades. Further, relations between CPS, general mental ability (g), academic achievement, and parental education were explored. Hungarian high school students in Grades 5 to 11 (N = 855) completed MicroDYN, which is a computer-based CPS test, and the Culture Fair Test 20-R as a measure of g. Results based on structural equation models showed that empirical modeling of CPS was in line with theories from cognitive psychology such that the two dimensions identified above were found in all grades, and that there was some development of CPS in school, although the Grade 9 students deviated from the general pattern of development. Finally, path analysis showed that CPS was a relevant predictor of academic achievement over and above g. Overall, results of the current study provide support for an understanding of CPS as a cross-curricular skill that is accessible through computer-based assessment and that yields substantial relations to school performance. Thus, the increasing attention CPS has currently received on an international level seems warranted given its high relevance for educational psychologists.

Computer-simulated scenarios have been part of psychological research on problem solving for more than 40 years. The shift in emphasis from simple toy problems to complex, more real-life oriented problems has been accompanied by discussions about the best ways to assess the process of solving complex problems. Psychometric issues such as reliable assessments and addressing correlations with other instruments have been in the foreground of these discussions and have left the content validity of complex problem solving in the background. In this paper, we return the focus to content issues and address the important features that define complex problems.