Measuring teamwork mental models to support training needs assessment, development, and evaluation: two empirical studies†

Abstract

SummaryThe present paper reports data from two studies that utilized a card sorting approach to measuring mental model similarity in naturalistic training environments. Results from the ®rst study indicated that higher ranking navy personnel held mental models of teamwork that were more similar to an empirically derived model of expert team performance than lower ranking personnel. Furthermore, comparisons of mental model similarity within groups of high and low ranking trainees and within groups of high and low experience trainees indicated greater similarity between those of higher rank and between those with greater experience. The second study tested the effects of a computer-based training (CBT) strategy that was designed to develop teamwork mental models that were more similar to the`expert model' described in Study 1. Using the same card sorting approach, positive training effects were demonstrated on similarity to the expert model, similarity to other trainees, and consistency.

The Mental Model Comparison of Expert and Novice Performance Improvement Practitioners

“…This is an interesting result since the current professional world greatly supports teamwork. Smith-Jentsch, Campbell, Milanovich, and Reynolds (2001) found that high-ranked navy personnel possessed teamwork mental models that were more parallel to the model, which was derived from the expert team, than those who were low-ranked. This study represents evidence between expert mental models and performance.…”

“…Every method cannot supersede one to another, and they have both strengths and weaknesses. The most commonly used methods in expertise research are: (a) interviews, such as unstructured, structured, one to one, and semi-structured (Hmelo-Silver & Pfeffer, 2004;Means & Voss, 1985;Nelson, 1989;Payne, 1991), (b) card-sorting (Chi & Koeske, 1983;Hodgkinson, 2002;Smith-Jentsch, et al, 2001), (c) thinking-aloud protocol Ericsson & Charness, 1994;Zahodnic, 2009), (d) analysis and completion of tasks and/or cases Nelson, 1989), (e) text comprehension (Burkhardt, Détienne, & Wiedenbeck, 1997), and (f) analysis of relatedness or proximity of concepts, such as link-weighted, that is, Pathfinder networks (Goldsmith & Johnson, 1990;Schvaneveldt, et al, 1985;Taricani & Clariana, 2006;Villachica, et al, 2001), and multidimensional scaling (Bradley, et al, 2006;McKeithen, et al, 1981;Schvaneveldt, et al, 1985). Rowe and Cooke (1995) studied a comparison of the effectiveness of three techniques, structured interviewing, related rating, and think-aloud protocol, on trouble-shooting performance related to airborne electronic equipment, and they found that relatedness rating via using PathFinder was the most effective technique.…”

Task conflict asymmetries: effects on expectations and performance

“…The logic provided is that for interdependent members working together to perform well, all involved must have a common understanding of the goals, the processes required to reach those goals, and the information that members have (Hinsz et al, 1997;Mohammed et al, 2000;Smith-Jentsch et al, 2001). We argue that if individuals working together have a common perception of the social processes in the dyad (i.e., conflict symmetry) it will be easier for them to work more effectively on their task (Hinsz et al, 1997;Marks et al, 2002;Mathieu et al, 2000).…”

Examining the convergent validity of shared mental model measures

“…The term mental model refers to organized knowledge structures, or sets of concepts and the associations among them (Langan-Fox, Code, & Langfield-Smith, 2000;Smith-Jentsch, Campbell, Milanovich, & Reynolds, 2001). Thus, mental models are defined in terms of both content knowledge and, importantly, the structure (or organization) of that content knowledge.…”