Why the Qualms With Qualitative? Utilizing Qualitative Methods in 360° Feedback

Abstract

Although the authors of the focal article provide a comprehensive definition of 360° feedback, one exclusionary criterion results in an overly narrow definition of 360° feedback. Specifically, Point 3 in their definition described the criticality of strictly using quantitative methods in collecting 360° feedback. The authors provided a brief rationale by stating, “Data generated from truly qualitative interviews would not allow comparisons between rater groups on the same set of behaviors” (Bracken, Rose, & Church, 2016, p. 765). Although there is little doubt about the value in taking a quantitative approach for gathering 360° feedback, it is not clear why this has to be the sole approach. Below, I outline three issues with taking this constricted methodology. That is, first, excluding qualitative methods is not in line with the purpose of 360° feedback, which is directed at minimizing criterion deficiency. Second, qualitative methodologies (in conjunction with quantitative methodologies) are more equipped to provide and inspire a call to action (supporting the change component addressed by the authors). Finally, there are qualitative methods that allow for rigorous quantitative analysis and can provide an additional source of macro organizational-level data.

Quantifying with words: An investigation of the validity of narrative‐derived performance scores

“…For example, receiving a below average rating on the dimension of adaptability, although succinct, is not incredibly informative. On the other hand, narrative comments allow one to comprehend the reasoning behind numerical ratings (Brutus, ; Kabins, ), providing rich data regarding a person's behaviors within specific contexts and how others perceive them. In this example, information such as “you failed to adopt usage of a new technology system and instead relied on old methods to get work done” would provide much richer information.…”

“…At the same time, narrative comments allow for the provision of additional construct‐relevant information not captured by traditional numerical ratings, thus also increasing the bandwidth of measurement. Given the free‐form nature of narratives, raters can elaborate more upon competencies, upon completion of goals, and how the behavior of the employee impacted others (Kabins, ). This might be particularly salient if traditional performance scales are overly broad, such as if managers only produce a single overall rating.…”

Statesmen or cheerleaders? Using topic modeling to examine gendered messages in narrative developmental feedback for leaders

“…Qualitative feedback is more likely to trigger behavior change than quantitative performance ratings (Kabins, 2016), especially when comments include prescriptive suggestions for development (Brutus, 2010). A survey of executives who received feedback consisting of both quantitative ratings and anonymous narrative comments found that these executives attended to narrative comments more than they did to numerical ratings (Ferstl and Bruskiewicz, 2000).…”

“…Brown, Kulik & Lim, 2016), there is a shortage of empirical research into the content of qualitative developmental feedback more broadly (i.e. what is said and how it is said: Kabins, 2016) and whether this differs for men and women in, or aspiring to, leadership roles. Instead, most research has focused on quantitative ratings of performance and issues such as rater agreement in feedback to leaders (Day, Fleenor, Atwater, Sturm & McKee, 2014;Markham, Markjam, & Smith, 2015).…”

“…As an inherently social process, leadership development takes place in an environment suffused with ideologies about what it means to be a leader and a woman (Ely et al, 2011). Despite claims that developmental feedback challenges self-perceptions, helps individuals locate areas for development, and contributes to leadership identity construction (Ely et al, 2011), remarkably little is understood about the mechanisms by which this type of feedback may result in change (Kabins, 2016), or how these may impact differentially for male and female leaders. Developmental feedback that is specific, constructive and challenging may still vary in how feasible it is to action, and the narrative format of developmental feedback provides opportunity for subtle messaging about desirable change for male and female leaders.…”