Communication evaluation and measurement: connecting research to practice

Buhmann, Alexander; Likely, Fraser; Geddes, David

doi:10.1108/jcom-12-2017-0141

Cited by 21 publications

(18 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite a 2012 AMEC European Summit presentation titled “The March to Standards” (Marklein and Paine, 2012), evidence shows achieving standardization within the industry in the years since these efforts has been challenging and seen mixed results (Buhmann et al , 2018; Paine, 2018). Buhmann et al state that “Standards development is advancing on two levels consistent with ISO recommendations: the level of technical standards for the measurement of communication, in other words, ‘how to standards,’ as in how to measure impressions and how to calculate ROI; and the level of process standards for evaluation as in the broader frameworks (of which measurement itself is just one important element)” (p. 115).…”

Section: Discussion: Maturity Models and Advancing Eandm Practice In Industrymentioning

confidence: 99%

Maturity as a way forward for improving organizations’ communication evaluation and measurement practices

Gilkerson

Swenson

Likely

2019

JCOM

Self Cite

View full text Add to dashboard Cite

Purpose The purpose of this paper is to propose an explication of the concept of “maturity,” as it applies to communication evaluation and measurement (E&M) practice, along with contextualization of recent maturity model adoption within academic and professional communities. Design/methodology/approach Drawing from previous work on maturity models within other fields, recent communication scholarship and industry practice, this paper fills a gap in the literature by offering a theoretical conceptualization of communication E&M maturity, including the construct’s core dimensions and sub-dimensions. Findings Communication E&M maturity is conceptualized into four essential elements: holistic approach, investment, alignment and culture. The contribution of E&M efforts is represented as the direct support of corporate strategy, and ultimately increased value, from the communications function. Operational elements of maturity include levels of analysis, time, budget, tools, skills, process, integration, motivations, relationships and standards. Originality/value In exploring the factors necessary for “mature” E&M programs, and specifically emphasizing the need for a holistic approach, along with sufficient investment and alignment, and conducive cultural factors, the research builds upon existing work examining how communication can serve to inform corporate strategy and create value for an organization. Greater understanding and application of the maturity concept has the potential to advance the field by increasing both accountability and credibility for the work done by the communications function.

show abstract

Section: Discussion: Maturity Models and Advancing Eandm Practice In Industrymentioning

confidence: 99%

Maturity as a way forward for improving organizations’ communication evaluation and measurement practices

Gilkerson

Swenson

Likely

2019

JCOM

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our protocol for message testing is not to be confused with broader campaign development and evaluation procedures, whether from communication and public health (Noar, 2009;Hornik, 2002;Hornik and Yanovitzky, 2003;Slater, 2006;Wakefield et al, 2010) or from public relations and strategic communication (Buhmann et al, 2018;Macnamara, 2018;Volk, 2016) wherein robust literature and debates reside. Message testing is often a part of campaign development and implementationsometimes ignored or neglectedbut functions more as a complement to the full dimensionality of campaign problem identification, planning, implementation, evaluation, feedback and re-planning with subsequent modification of interventions.…”

Section: Limitations and Counterargumentsmentioning

confidence: 99%

Reliable, valid and efficient evaluation of media messages

Kim

Cappella

2019

JCOM

View full text Add to dashboard Cite

Purpose In the field of public relations and communication management, message evaluation has been one of the starting points for evaluation and measurement research at least since the 1970s. Reliable and valid message evaluation has a central role in message effects research and campaign design in other disciplines as well as communication science. The purpose of this paper is to offer a message testing protocol to efficiently acquire valid and reliable message evaluation data. Design/methodology/approach A message testing protocol is described in terms of how to conceptualize and evaluate the content and format of messages, in terms of procedures for acquiring and testing messages and in terms of using efficient, reliable and valid measures of perceived message effectiveness (PME) and perceived argument strength (PAS). The evidence supporting the reliability and validity of PME and PAS measures is reviewed. Findings The message testing protocol developed and reported is an efficient, reliable and valid approach for testing large numbers of messages. Research limitations/implications Researchers’ ability to select candidate messages for subsequent deeper testing, for various types of communication campaigns, and for research in theory testing contexts is facilitated. Avoiding the limitations of using a single instance of a message to represent a category (also known as the case-category confound) is reduced. Practical implications Communication campaign designers are armed with tools to assess messages and campaign concepts quickly and efficiently, reducing pre-testing time and resources while identifying “best-in-show” examples and prototypes. Originality/value Message structures are conceptualized in terms of content and format features using theoretically driven constructs. Measures of PAS and PME are reviewed for their reliability, construct and predictive validity, finding that the measures are acceptable surrogates for actual effectiveness for a wide variety of messages and applications. Coupled with procedures that reduce confounding by randomly nesting messages within respondents and respondents to messages, the measures used and protocol deployed offer an efficient and utilitarian approach to message testing and modeling.

show abstract

“…The design of communication campaigns is a complicated and multidimensional undertaking whether for public health goals (Slater, 2006) or for public relations or strategic communication goals (Buhmann, Likely, & Geddes, 2018;Macnamara, 2018). One typical component of communication campaigns is message selection (Kim & Cappella, in press).…”

Section: An Experimental Study Of Recommendation Algorithms For Tailomentioning

confidence: 99%

An Experimental Study of Recommendation Algorithms for Tailored Health Communication

Kim¹,

Yang²,

Kim³

et al. 2019

Preprint

View full text Add to dashboard Cite

Recommendation algorithms are widely used in online cultural markets to provide personalized suggestions for products like books and movies. At the heart of the commercial success of recommendation algorithms is their ability to make an accurate prediction of a target person’s preferences for previously unseen items. Can these algorithms also be used to predict which health messages an individual will evaluate favorably, and thereby provide effective tailored communication to the person? Although there is evidence that message tailoring enhances persuasion, little research has examined the effectiveness of recommendation algorithms for tailored health interventions aimed at promoting behavior change. We developed a message tailoring algorithm to select smoking-related public service announcements (PSAs) for smokers, and experimentally test its effectiveness in predicting a target smoker’s evaluations of PSAs and encouraging smoking cessation. The tailoring algorithm was constructed using multiple levels of data on smokers’ PSA rating history, individual differences, content features of the PSAs, and other smokers’ PSA ratings. We conducted a longitudinal online experiment to examine its efficacy in comparison to two non-tailored methods: “best in show” (choosing messages most preferred by other smokers) and “off the shelf” (random selection from eligible ads). The results showed that the tailoring algorithm produced more accurate predictions of smokers’ message evaluations than the simple-average method used for the “best in show” approach. Smokers who viewed PSAs recommended by the tailoring algorithm were more likely than those receiving a random set to evaluate the PSAs favorably and quit smoking. There was no significant difference between the “best in show” and “off the shelf” methods in message assessment and quitting behavior.

show abstract

Communication evaluation and measurement: connecting research to practice

Cited by 21 publications

References 0 publications

Maturity as a way forward for improving organizations’ communication evaluation and measurement practices

Maturity as a way forward for improving organizations’ communication evaluation and measurement practices

Reliable, valid and efficient evaluation of media messages

An Experimental Study of Recommendation Algorithms for Tailored Health Communication

Contact Info

Product

Resources

About