An Objective Approach to Identifying Diagnostic Expertise Among Power System Controllers

Abstract

Objective: The present study investigated whether performance across a range of cue-based cognitive tasks differentiated the diagnostic performance of power control operators into three distinct groups, characteristic of novice, competence, and expertise. Background: Despite its increasing importance in the contemporary workplace, there is little understanding of the cognitive processes that distinguish novice, competent, and expert performance in the context of remote diagnosis. However, recent evidence suggests that cue acquisition and utilization may represent a mechanism by which the transition from novice to expertise occurs. Method: The study involved the application of four distinct cue-based tasks within the context of power system control. A total of 65 controllers, encompassing a range of industry experience, completed the tasks as part of an in-service training program. Results: Using a cluster analysis, it was possible to extract three distinct groups of operators on the basis of their performance in the cue-based tasks, and these groups corresponded to differences in diagnostic performance. Conclusion: The results indicate assessments of the capacity to extract and utilize cues were able to distinguish expert from competent practitioners in the context of power control. Application: Assessments of the capacity to extract and utilize cues may be used in the future to distinguish expert from nonexpert practitioners, particularly in the context of remote diagnosis.

Trait-based cue Utilization and initial skill acquisition: implications for models of the progression to expertise

“…These typologies were based on the outcomes of the EXPERTise tasks and were employed in this case due to the correspondence with previous methodological approaches to the application of EXPERTise-related outcomes (Loveday et al, 2013b, c; Loveday and Wiggins, 2014; Wiggins et al, 2014). The calculation of typologies began with the aggregation of the responses within the tasks, the calculation of z -scores, and a cluster analysis to identify whether two, meaningful typologies could be established.…”

“…The construct validity of EXPERTise has been demonstrated in a number of different domains, whereby typologies that were formed on the basis of performance across the EXPERTise tasks differentiated both simulated and actual performance in the workplace (Loveday et al, 2013b, c; Loveday and Wiggins, 2014; Wiggins et al, 2014). The test–retest reliability of the typologies has been demonstrated in the context of power control operators at six monthly intervals, κ = 0.59 (Loveday et al, 2013a).…”

Cue Utilization and Broad Indicators of Workplace Expertise

“…This is a notable outcome since previous investigations of recognition-primed decision making have associated cue utilization only with decision-making performance during discrete, simulated tasks that may not have translated to actual performance in the domain (Klein, 1989;Loveday, Wiggins, Harris, et al, 2013;Loveday, Wiggins, Searle, et al, 2013).…”

“…An example FDT scenario is provided in Appendix C. Given evidence that cue utilization requires the ability to discriminate more informative from less informative features of the domain (Brunswik, 1955), Weiss and Shanteau (2003) proposed that differential evaluations of the perceived utility of individual features during a scenario is indicative of expertise. This ability can be expressed mathematically as the variance of the participants' ratings of feature utility in the feature discrimination task, whereby greater variance is indicative of more discriminant ratings of utility between cues within the scenario (Loveday, Wiggins, Harris, et al, 2013;Pauley, O'Hare, & Wiggins, 2009;Schvaneveldt et al, 2001;Weiss & Shanteau, 2003;Wiggins & O'Hare, 2003a).…”

Injury Risk Estimation Expertise

“…Subjective rating of cue utility has been used in other expertise studies such as the feature discrimination task. [24][25][26]46 To estimate the potential relationship of general cognitive abilities to overall test performance and, specifically, to assess discriminant validity, participants completed the Berlin Numeracy Test (BNT). The BNT has been extensively validated for assessment of statistical numeracy and risk literacy, which is the ability to accurately interpret and make good decisions based on information about risk.…”