Using Crowd-Sourced Speech Data to Study Socially Constrained Variation in Nonmodal Phonation

Gittelson, Benjamin William; Leemann, Adrian; Tomaschek, Fabian

doi:10.3389/frai.2020.565682

Cited by 7 publications

(5 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In research, ASR, and forced alignment, have already proven extremely useful in the field of phonetics, sociophonetics and speech science more generally (some examples are Gonzalez et al, 2017;Mackenzie and Turton, 2020;Villarreal et al, 2020;Gittelson et al, 2021). Kisler et al (2017) describe the "paradigm shift" that has occurred over recent years due to internet speed and connections being vastly improved, now allowing webbased platforms to be accessed and used easily by researchers.…”

Section: Introductionmentioning

confidence: 99%

Does Automatic Speech Recognition (ASR) Have a Role in the Transcription of Indistinct Covert Recordings for Forensic Purposes?

Loakes

2022

Front. Commun.

View full text Add to dashboard Cite

The transcription of covert recordings used as evidence in court is a huge issue for forensic linguistics. Covert recordings are typically made under conditions in which the device needs to be hidden, and so the resulting speech is generally indistinct, with overlapping voices and background noise, and in many cases the acoustic record cannot be analyzed via conventional phonetic techniques (i.e. phonetic segments are unclear, or there are no cues at all present acoustically). In the case of indistinct audio, the resulting transcripts that are produced, often by police working on the case, are often questionable and despite their unreliable nature can be provided as evidence in court. Injustices can, and have, occurred. Given the growing performance of automatic speech recognition (ASR) technologies, and growing reliance on such technologies in everyday life, a common question asked, especially by lawyers and other legal professionals, is whether ASR can solve the problem of what was said in indistinct forensic audio, and this is the main focus of the current paper. The paper also looks at forced alignment, a way of automatically aligning an existing transcriptions to audio. This is an area that needs to be explored in the context of forensic linguistics because transcripts can technically be “aligned” with any audio, making it seem as if it is “correct” even if it is not. The aim of this research is to demonstrate how automatic transcription systems fare using forensic-like audio, and with more than one system. Forensic-like audio is most appropriate for research, because there is greater certainty with what the speech material consists of (unlike in forensic situations where it cannot be verified). Examples of how various ASR systems cope with indistinct audio are shown, highlighting that when a good-quality recording is used ASR systems cope well, with the resulting transcript being usable and, for the most part, accurate. When a poor-quality, forensic-like recording is used, on the other hand, the resulting transcript is effectively unusable, with numerous errors and very few words recognized (and in some cases, no words recognized). The paper also demonstrates some of the problems that arise when forced-alignment is used with indistinct forensic-like audio—the transcript is simply “forced” onto an audio signal giving completely wrong alignment. This research shows that the way things currently stand, computational methods are not suitable for solving the issue of transcription of indistinct forensic audio for a range of reasons. Such systems cannot transcribe what was said in indistinct covert recordings, nor can they determine who uttered the words and phrases in such recordings, nor prove that a transcript is “right” (or wrong). These systems can indeed be used advantageously in research, and for various other purposes, and the reasons they do not work for forensic transcription stems from the nature of the recording conditions, as well as the nature of the forensic context.

show abstract

Section: Introductionmentioning

confidence: 99%

Does Automatic Speech Recognition (ASR) Have a Role in the Transcription of Indistinct Covert Recordings for Forensic Purposes?

Loakes

2022

Front. Commun.

View full text Add to dashboard Cite

show abstract

“…There is some work on the automated study of laryngeal features. Gittelson et al (2021) used the English Dialects App (Leemann, Kolly, & Britain, 2018) and crowd-sourced audio from 2500 English speakers, aligned it with MAUS, and found age and gender effects on breathy and creaky voice. Tavi et al (2019) trained a CNN to find creaky voice in 30 emergency calls in Finnish.…”

Section: Examples Of Computational Sociophonetic Researchmentioning

confidence: 99%

“…Gittelson et al. (2021) used the English Dialects App (Leemann, Kolly, & Britain, 2018) and crowd‐sourced audio from 2500 English speakers, aligned it with MAUS, and found age and gender effects on breathy and creaky voice. Tavi et al.…”

Section: Examples Of Computational Sociophonetic Researchmentioning

confidence: 99%

Computational sociophonetics using automatic speech recognition

Coto-Solano

2022

Language and Linguist. Compass

View full text Add to dashboard Cite

Recent years have seen numerous advances in natural language processing that can help accelerate sociophonetic work. These include software to align speech recordings with their transcriptions, as well as to transcribe audio automatically. This solves a major bottleneck and will help process larger datasets and test hypotheses more efficiently. This paper will summarise recent progress, highlight relevant examples of sociophonetic research, and comment on the technical and ethical issues at the cutting edge of natural language processing.

show abstract

“…H1-H2 is highly correlated with the degree of glottal constriction, and it has more cycle-to-cycle variability in dysphonic voices than in non-pathologic voices (Urberg-Carlson, 2013). Interestingly, lower H1-H2 has been associated with creaky voice, while higher H1-H2 occurs with breathy voice [ (Gittelson et al, 2021;Keating and Esposito, 2006); see Figure 2]. This pattern may correspond to the current findings, as creakiness was found to be one of the most robust elements of expressions of pain and discomfort (Jenkins and Hepburn, 2015), possibly requiring less effort to produce, and a by-product of not fully opening one's throat and engaging vocal folds.…”

Section: Other Acoustic Measuresmentioning

confidence: 99%

Can the Vocal Expression of Intellectually Disabled Individuals Be Used as a Pain Indicator? Initial Findings Supporting a Possible Novice Assessment Method

Icht

Ressis-tal²,

Lotan

2021

Front. Psychol.

View full text Add to dashboard Cite

Pain is difficult to assess in non-verbal populations such as individuals with intellectual and developmental disability (IDD). Due to scarce research in this area, pain assessment for individuals with IDD is still lacking, leading to maltreatment. To improve medical care for individuals with IDD, immediate, reliable, easy to use pain detection methods should be developed. The goal of this preliminary study was to examine the sensitivity of acoustic features of vocal expressions in identifying pain for adults with IDD, assessing their feasibility as a pain detection indicator for those individuals. Such unique pain related vocal characteristics may be used to develop objective pain detection means. Adults with severe-profound IDD level (N = 9) were recorded in daily activities associated with pain (during diaper changes), or without pain (at rest). Spontaneous vocal expressions were acoustically analyzed to assess several voice characteristics. Analyzing the data revealed that pain related vocal expressions were characterized by significantly higher number of pulses and higher shimmer values relative to no-pain vocal expressions. Pain related productions were also characterized by longer duration, higher jitter and Cepstral Peak Prominence values, lower Harmonic-Noise Ratio, lower difference between the amplitude of the 1st and 2nd harmonic (corrected for vocal tract influence; H1H2c), and higher mean and standard deviation of voice fundamental frequency relative to no-pain related vocal productions, yet these findings were not statistically significant, possibly due to the small and heterogeneous sample. These initial results may prompt further research to explore the possibility to use pain related vocal output as an objective and easily identifiable indicator of pain in this population.

show abstract

Using Crowd-Sourced Speech Data to Study Socially Constrained Variation in Nonmodal Phonation

Cited by 7 publications

References 38 publications

Does Automatic Speech Recognition (ASR) Have a Role in the Transcription of Indistinct Covert Recordings for Forensic Purposes?

Does Automatic Speech Recognition (ASR) Have a Role in the Transcription of Indistinct Covert Recordings for Forensic Purposes?

Computational sociophonetics using automatic speech recognition

Can the Vocal Expression of Intellectually Disabled Individuals Be Used as a Pain Indicator? Initial Findings Supporting a Possible Novice Assessment Method

Contact Info

Product

Resources

About