Extending the Assessment of Complex Problem Solving to Finite State Automata

Abstract

Abstract. Recent advancements in the assessment of Complex Problem Solving (CPS) build on the use of homogeneous tasks that enable the reliable estimation of CPS skills. The range of problems featured in established instruments such as MicroDYN is consequently limited to a specific subset of homogeneous complex problems. This restriction is problematic when looking at domain-specific examples of complex problems, which feature characteristics absent from current assessment instruments (e.g., threshold states). We propose to utilize the formal framework of Finite State Automata (FSA) to extend the range of problems included in CPS assessment. An approach based on FSA, called MicroFIN, is presented, translated into specific tasks, and empirically investigated. We conducted an empirical study (N = 576), (1) inspecting the psychometric features of MicroFIN, (2) relating it to MicroDYN, and (3) investigating the relations to a measure of reasoning (i.e., CogAT). MicroFIN (1) exhibited adequate measurement characteristics and multitrait-multimethod models indicated (2) the convergence of latent dimensions measured with MicroDYN. Relations to reasoning (3) were moderate and comparable to the ones previously found for MicroDYN. Empirical results and corresponding explanations are discussed. More importantly, MicroFIN highlights the feasibility of expanding CPS assessment to a larger spectrum of complex problems.

Complex Problems in Entrepreneurship Education: Examining Complex Problem‐Solving in the Application of Opportunity Identification

“…Based on this framework, MicroFIN aims to counter limitations in the breadth of problems included in previous CPS instruments (e.g., MicroDYN; [56]) and facilitate a greater heterogeneity in tasks [49,50,57]. MicroFIN was recently found to have convergent validity with an established CPS instrument and discriminant validity with different measures of general mental ability (GMA; [49,58]). MicroFIN tasks share a general layout of input variables that influence output variables and are in accordance with the theoretical background as outlined in the Introduction (i.e., test items contain (a) values that change with the user's interaction and (b) various nontransparent interactions between variables, such as threshold or equilibrium states in the input variables; [13]).…”

“…However, VOTAT is not sufficient to solve a complex problem that requires a whole set of strategies and their adaptive use (see [48]). The VOTAT strategy is therefore a specific one among many different exploration strategies that might help to simplify knowledge acquisition in the real world as well as in current CPS tests that have been applied in the present study [49,50].…”

“…The goal in this nontransparent task was to balance the interests of various parties (e.g., families and industries) by improving their Figure 1: Screenshot of the MicroFIN item "Plan-o-Maton" [49,50]. Problem solvers have to balance the interests of various parties in a city by making alterations in the urban landscape.…”

“…We employed one introduction task and a set of six complex task simulations from the fully computer-based CPS assessment test MicroFIN [49,50]. MicroFIN features multiple, dynamic tasks that are based on a formal framework called "Finite State Automata" [55].…”

“…MicroFIN features multiple, dynamic tasks that are based on a formal framework called "Finite State Automata" [55]. Based on this framework, MicroFIN aims to counter limitations in the breadth of problems included in previous CPS instruments (e.g., MicroDYN; [56]) and facilitate a greater heterogeneity in tasks [49,50,57]. MicroFIN was recently found to have convergent validity with an established CPS instrument and discriminant validity with different measures of general mental ability (GMA; [49,58]).…”