The relationship between trained ratings and untrained listeners’ judgments of global coherence in extended monologues

Abstract

Background: Global coherence rating scales have been used by a number of researchers to examine spoken discourse in populations with and without acquired neurogenic communication disorders. The 4-point global coherence (GC) scale in the current study has demonstrated reliability and convergent validity. However, we have not yet established how a global coherence rating corresponds to functional communication. Aims:The current study explores the clinical meaningfulness of the 4-point GC rating scale. Survey questions and ratings were developed to examine discourse quality and functional coherence as perceived by untrained listeners. Ratings by untrained listeners were compared to trained discourse ratings using the established 4-point GC scale.Methods: Twelve discourse samples, scored by a trained rater, were selected for the current study from a previously collected set of discourse transcripts. Transcripts were extended monologues in response to one of four possible open-ended questions that were re-recorded by the trained rater to remove any distracting features of the original recordings, such as articulatory errors. Twenty-four untrained listeners rated the discourse samples using a short questionnaire that included questions for each sample about: topic maintenance, inclusion of unnecessary information, and the listener level of interest and attention. Questions for untrained listeners were based on operational definitions of global coherence and discourse quality respectively. These untrained ratings were compared to trained ratings of global coherence. Outcome measures were compared using nonparametric tests and a Spearman Rank Order test was also used to examine relationships among variables.Outcomes and Results: Untrained listeners' ratings for topic maintenance and inclusion of unnecessary information were significantly different between trained low, medium, and high global coherence ratings. Topic maintenance and inclusion of unnecessary information were also