How to assess team performance in terms of control: a protocol based on cognitive systems engineering

Abstract

This article presents a protocol for assessing team performance in terms of control based on cognitive systems engineering theory. The protocol changes the focus of team performance assessment from good and bad behaviours towards team activity. By using Hollnagel's Contextual Control Model (COCOM), a protocol is developed so that team activity can be described as four control modes. Data is collected through observation and questionnaires and is analysed in time intervals. Each time interval is then given a control mode. Based on how the control mode changes over time the performance of the team can be assessed. The protocol was tested in a pilot study where commanding staff exercises were analysed. The results show that the protocol can be used to identify differences in team performance. The study also gives empirical validity to COCOM in that the loss of control in emergencies corresponds to an opportunistic or a scrambled control mode.

From Crew Resource Management To Operational Resilience

“…In this sense, control 'happens' during the interaction of "humantask-artifact" and is goal-oriented and influenced by the context in which the situated activity happens. When developing methods for team performance assessment Hollnagel's model of contextual control modes (Hollnagel & Woods, 2005) has been operationalized to map how the perceived level of control shifts during a scenario by including both the participants' own reflections and the observer's interpretations (Palmqvist, Bergström, Henriqson, 2011). Noteworthy is that no level of control is seen as more appropriate than another, but instead is highly contingent upon the situation and context.…”

Bridging the Macro and the Micro by Considering the Meso: Reflections on the Fractal Nature of Resilience

“…Interest in how the ability to adapt to unexpected and escalating situations varies with different levels of experience in both the domain of working and that of crisis management. Studies have been conducted focusing on how teams reached different levels of control based on how they coordinated their actions (Bergström et al 2010, Palmqvist et al 2012. The notion of coordination provides an analytic language locating the target of analysis at the functional level of the system studied.…”

“…It is operationalized as the result of different actors' ability to predict the behavior of each other, i.e., interpredictability, their common terminology, values and frameworks, i.e., common ground, their ways to exercise influence on each other, i.e., directability, together with their previous experiences, or assumptions, of working together, i.e., choreography (Klein et al 2004). It has been suggested that the macro cognitive framework is a promising one, and in further studies the language of control has been operationalized to map and follow the performance of a crisis management team, again based on their collective rather than individual performances (Bergström et al 2010, Palmqvist et al 2012). …”