Inter-transcriber reliability of toBI prosodic labeling

Syrdal, Ann K.; McGory, Julia Tevis

doi:10.21437/icslp.2000-521

Cited by 50 publications

(20 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transcription systems for English and German based on autosegmental-metrical phonology make a distinction between H* and L+H*. Yet, inter-transcriber reliability is generally quite low for these accents (Grice et al, 1996;Pitrelli et al, 1994;Syrdal & McGory, 2000) and empirical evidence for their distinction is scarce and controversial (Arvaniti et al, 2022;Ladd & Schepman, 2003). A similar problem applies to the distinction between H+!H* and H+L* (Grice et al, 2009;Rathcke & Harrington, 2010).…”

Section: Categorical Distinctions Of Prominence Relationsmentioning

confidence: 99%

The inverse relation of pre-nuclear and nuclear prominences in German

Roessig¹

2023

Preprint

View full text Add to dashboard Cite

The importance of pre-nuclear prominences for focus marking has been largely neglected. Recent studies however present first evidence that the prosody of the pre-nuclear region indicates its status as part of a broad focus or as pre-focal. This study presents a systematic investigation of the pre-nuclear domain and its relation to the nuclear accent in German. The results show that the realization of the pre-nuclear domain indeed depends on whether it is focal or pre-focal: The pre-nuclear noun is characterized by larger F0 excursions, higher F0 maxima and larger durations when it is in broad focus than when it precedes a narrow focus. Furthermore, the realization of a pre-focal, pre-nuclear domain depends on the following focus: The pre-nuclear noun is produced with smaller F0 excursions, lower F0 maxima and shorter durations before a corrective focus than before a non-corrective narrow focus. The comparison to the nuclear accent suggests an inverse relationship, i.e., the pre-nuclear domain becomes less prominent when the nuclear domain becomes more prominent. The findings suggest that the phonetic manifestation of information structure is distributed over larger prosodic domains and entails the modulation of phrasal prominence profiles rather than just local adjustments of the nucleus.

show abstract

Section: Categorical Distinctions Of Prominence Relationsmentioning

confidence: 99%

The inverse relation of pre-nuclear and nuclear prominences in German

Roessig¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, emphasizing the very high agreement on the presence versus absence of edge tones and pitch accents obscures the much lower agreement on the type of intonational label that should be assigned to each of these events. In [20], for example, pairwise agreement on the specific pitch accent label failed to exceed 50% for six of the eight label types. Likewise, agreement on the specific edge tone label failed to exceed 50% for six of the nine label types.…”

Section: Inter-transcriber Agreementmentioning

confidence: 96%

“…The reliability studies have shown remarkable consistency in their results: Even though different researchers have conducted them with different labelers and on different speech materials, pairwise agreement between labelers on the presence versus absence of an edge tone (either phrase accent or boundary tone) has ranged from 85% in [15], to 86% in [7], and 92% in [20]. Similarly, pairwise agreement on the presence versus absence of a prominence (pitch accent) has ranged from 81% in [15], to 87% in [7], and 91% in [20].…”

Section: Inter-transcriber Agreementmentioning

confidence: 97%

“…The extent to which this was possible was, in fact, used to help guide the development of ToBI [15]. More recently, this study has been supplemented by [20], in which the labeling done by five graduate students and one postdoctoral researcher, all working in the same lab with the same training, was carefully analyzed. There have also been studies of inter-transcriber agreement in other languages, notably German [7].…”

Section: Inter-transcriber Agreementmentioning

confidence: 99%

See 1 more Smart Citation

ToBI or not toBI?

Wightman

2002

Speech Prosody 2002

View full text Add to dashboard Cite

In the decade that has passed since the introduction of the ToBI system for the transcription of prosody, speech technology has moved out of the laboratory and into commercial applications on several fronts. However, virtually none of the commercial products have made large-scale use of prosody. Nevertheless, researchers in both recognition and synthesis continue to agree that better utilization of prosody is essential to improving the performance and acceptability of commercial systems. In this paper, we review the current state of prosody in commercial systems, and examine how the ongoing discussions related to what and how to transcribe with respect to prosody have simultaneously advanced and inhibited the field. In particular, we argue that, in hindsight, the ToBI system contains several flaws that have limited its acceptance and application.

show abstract

“…Though Information Structure theory can give us an idea of prosodic expectations, prosody perception is known for high inter-listener variability [19,20] even with expert training [24]. So, the fact that the RPT framework was specifically developed to capture variability in prosody perception from non-expert listeners makes it a natural choice for exploring the perception of synthesized speech.…”

Section: Introductionmentioning

confidence: 99%

Location, Location: Enhancing the Evaluation of Text-to-Speech synthesis using the Rapid Prosody Transcription Paradigm

Gutierrez¹,

Oplustil-Gallegos²,

Lai³

2021

11th ISCA Speech Synthesis Workshop (SSW 11)

View full text Add to dashboard Cite

Text-to-Speech synthesis systems are generally evaluated using Mean Opinion Score (MOS) tests, where listeners score samples of synthetic speech on a Likert scale. A major drawback of MOS tests is that they only offer a general measure of overall quality-i.e., the naturalness of an utterance-and so cannot tell us where exactly synthesis errors occur. This can make evaluation of the appropriateness of prosodic variation within utterances inconclusive. To address this, we propose a novel evaluation method based on the Rapid Prosody Transcription paradigm. This allows listeners to mark the locations of errors in an utterance in real-time, providing a probabilistic representation of the perceptual errors that occur in the synthetic signal. We conduct experiments that confirm that the fine-grained evaluation can be mapped to system rankings of standard MOS tests, but the error marking gives a much more comprehensive assessment of synthesized prosody. In particular, for standard audiobook test set samples, we see that error marks consistently cluster around words at major prosodic boundaries indicated by punctuation. However, for question-answer based stimuli, where we control information structure, we see differences emerge in the ability of neural TTS systems to generate context-appropriate prosodic prominence.

show abstract

Inter-transcriber reliability of toBI prosodic labeling

Cited by 50 publications

References 0 publications

The inverse relation of pre-nuclear and nuclear prominences in German

The inverse relation of pre-nuclear and nuclear prominences in German

ToBI or not toBI?

Location, Location: Enhancing the Evaluation of Text-to-Speech synthesis using the Rapid Prosody Transcription Paradigm

Contact Info

Product

Resources

About