The status quo of evaluation in public diplomacy: insights from the US State Department

Abstract

Purpose In recent years, expectations for demonstrating the impact of public diplomacy programs have dramatically increased. Despite increased calls for enhanced monitoring and evaluation, what texts exist on the subject suggest the state of practice is grim. However, while the current debate is based mostly on practice reports, conceptual work from academics or anecdotal evidence, we are missing empirical insights on current views of monitoring and evaluation from practitioners. Such a practice-level perspective is central for better understanding factors that may actually drive or hamper performance evaluation in day-to-day public diplomacy work. The purpose of this paper is to update knowledge on the state of evaluation practice within public diplomacy from the perspectives of practitioners themselves. Design/methodology/approach This study assesses the state of evaluation in public diplomacy through qualitative interviews with public diplomacy officers working for the US Department of State – a method heretofore unused in studies of the topic. In total, 25 in-depth interviews were conducted with officers in Washington, DC and at posts around the world. Findings The interviews suggest that practitioners see evaluation as underfunded despite increased demands for accountability. Further, the results show a previously not discussed tension between diplomacy practitioners in Washington, DC and those in the field. Practitioners are also unclear about the goals of public diplomacy, which has implications for the enactment of targeted evaluations. Originality/value The research uncovers the perceptions of evaluation from the voices of those who must practice it, and elaborates on the common obstacles in the enactment of public diplomacy, the influence of multiple actors and stakeholders on evaluation practice, as well as the perceived goals of public diplomacy programming. No empirical research has considered the state of evaluation practice. Moreover, the study uses qualitative interview data from public diplomacy officers themselves, an under-used method in public diplomacy research. The findings provide insights that contribute to future public diplomacy strategy and performance management.

Drivers and barriers in public diplomacy evaluation: understanding attitudes, norms, and control

“…Other than in quantitative applications, where the TPB is tested for predicting concrete and time-bound behaviors, the explorative, qualitative setting of this study led us to inquire about evaluation as a domain of behavior, or ‘aggregates of specific behaviors’ (Ajzen, 1991) related to performance evaluation. This also acknowledges that public diplomacy is still not a clearly defined practice and, hence, there are few common ideas about public diplomacy goals (Sommerfeldt and Buhmann, 2019) and appropriate routes to their evaluation (Sevin, 2017).…”

“…Despite the difficulties in evaluating public diplomacy programs, calls for accountability and more sophisticated systems for evaluation have dramatically increased in many countries (Cull, 2014). While comparative research shows that "demands for accountability and value-for-money" are significantly reshaping public diplomacy practice around the world (Pamment, 2012a, p. 333), recent literature on monitoring and evaluation (M&E) still paints a picture of low satisfaction with the state of the field (e.g., ACPD, 2014ACPD, , 2018Banks, 2011;Gonzales, 2015;Sommerfeldt and Buhmann, 2019). Indeed, for most public diplomacy practitioners, M&E remains a "daunting task" (Sevin, 2017).…”

New avenues in communication evaluation and measurement (E&M): towards a research agenda for the 2020s

“…case-category confound). Ultimately, the contribution also shows that strong inspiration that can be drawn from adjacent fields, such as health communication, that have developed very sophisticated approaches with high potential for adaptation within the domain of communication E&M. Sommerfeldt and Buhmann (2019) posit that, despite increased calls for enhanced monitoring and evaluation in public diplomacy, the state of practice remains grim. To better understand this situation, they undertook a qualitative study uncovering the perceptions around evaluation from the voices of those who must practice it.…”

“…Their findings imply that future investigations into E&M barriers need to consider further contextual factors and go beyond self-disclosed surveys of communication professionals. Sommerfeldt and Buhmann (2019) add to the research on barriers from a much-needed qualitative perspective, producing insights form the PR-related field of public diplomacy. Their findings highlight resemblances with studies conducted in the professional fields of strategic communication and public relations more generally, and they point out specifically to so far overlooked tensions regarding E&M between central and peripheral units of organisation.…”

Why and How Do Sojourners Talk about Macao? Effects of Perceived Risk and Expected Benefit

“…Public diplomacy literature has been dominated by heuristic, normative and prescriptive studies. There have been numerous calls for the development of frameworks for evaluating public diplomacy effectiveness (Banks 2011;Hayden 2017;Pahlavi 2007;Pamment 2014;Sommerfeldt and Buhmann 2019). Recently, some studies have attempted to evaluate and assess public diplomacy outcomes and/or country images by listening to what foreign publics say about countries (Ingenhoff and Chariatte 2019;Ingenhoff, White, Buhmann and Kiousis 2019;Sevin and Ingenhoff 2018).…”