Measuring Team Performance in Complex and Dynamic Military Environments: The SPOTLITE Method

A literature review of methods for providing enhanced operational oversight of teams in emergency management

“…Many of the methods used to monitor teams require careful observation and detailed coding of observations, which takes time and requires access to team-specific information. In a fast-paced context observers are likely to experience considerable difficulty in paying close attention to all team members' actions while simultaneously recording their observations (MacMillan et al, 2013). These limitations may also be particularly problematic for people who are not co-located with a team and who therefore do not have the ability to directly observe how that team is functioning.…”

“…These limitations may also be particularly problematic for people who are not co-located with a team and who therefore do not have the ability to directly observe how that team is functioning. MacMillan et al (2013) suggest two ways of addressing the issues associated with the ability of observers to code aspects of teamwork in real-time. They suggested that: coding should be reviewed at a later stage during the team debriefing, or that observers should complete an additional instrument after the scenario and compare to the results gained during the real-time coding.…”

“…For example, the scenario-based performance observation tool for learning in team environments (SPOTLITE) uses behavioural markers to allow observers to measure team performance. MacMillan et al (2013) reported on the problem of identifying team readiness for active F-16 combat fighter pilots because of the complexity of the environment. In this study, each team consisted of four pilots each with an observable cockpit display that required the absolute attentive focus of the observers (MacMillan et al., 2013).…”

Team Performance: Evidence for Validity of a Measure

“…In this regard, Brannick and Prince (2009) point out that the measures available usually focus on the evaluation of concrete results such as the number of takeoffs and landings made without incident, in the case of flight crews, or successful surgical procedures, in the case of medical teams. When the emphasis is on judgmental assessment of results, review of the area literature even shows the marked presence of customized tools, as in the study by McMillan et al (2013) that describes the process of developing a measure to assess the performance of teams of pilots. These findings support the relevance of the study carried out, focused on the development of a non-specific measure, applicable to different organizational contexts, and that can be answered both by the members themselves as well as by the team supervisor.…”

“…They attribute this, in large part, to the measurement instruments, which are described as scarce, and to the fact that organizations tend to develop customized scales that capture only their own reality. Focusing on the specificity that characterizes these measures, McMillan, Entin, Morley, and Bennett Jr. (2013) discuss the existence of instruments tailored to specific scenarios, and following the central trend of this field, propose a measure for use in military environments, applicable to teams composed of four members, F16 aircraft pilots. From these findings, it is concluded that the development of a measure that facilitates team performance assessment, for cross-organizational use, applicable to teams that perform many different tasks, is an important contribution.…”