Audio-Tokens: A toolbox for rating, sorting and comparing audio samples in the browser

Abstract

Here we describe a JavaScript toolbox to perform online rating studies with auditory material. The main feature of the toolbox is that audio samples are associated with visual tokens on the screen that control audio playback and can be manipulated depending on the type of rating. This allows the collection of single- and multidimensional feature ratings, as well as categorical and similarity ratings. The toolbox (github.com/pwdonh/audio_tokens) can be used via a plugin for the widely used jsPsych, as well as using plain JavaScript for custom applications. We expect the toolbox to be useful in psychological research on speech and music perception, as well as for the curation and annotation of datasets in machine learning.

Aberrant neurophysiological signaling associated with speech impairments in Parkinson’s disease

“…We used a novel approach to quantify speech intelligibility along three features (i.e., voice, articulation, and prosody) as implemented in the audio_tokens toolbox (Fig. 1A ) 74 . Speech quality assessments were consistent across raters for all assessed features: voice (intra-class correlation coefficient [ICC] = 0.89, 95% CI = [0.83 0.93]), articulation (ICC = 0.88, 95% CI = [0.81 0.92]), and prosody (ICC = 0.78, 95% CI = [0.66 0.86]; Fig.…”

“…Our approach to speech impairment quantification is predicated on the notion that clinical endpoints for speech production studies in PD need to improve speech perception by human raters [17][18][19]75,76 . One novel contribution of the present study is the rating of speech features by non-experts through an intuitive and interactive app (audio_tokens) 74 , that facilitates simultaneous comparison of speech samples from multiple study participants. Despite being obtained from non-experts, the ratings exhibited a degree of inter-rater reliability comparable to those from gold-standard procedures in the field, such as speech intelligibility quantification using transcription 76 .…”

“…The speech features of the recordings were then quantified in two ways: (1) using an automated extraction approach based on commonly-used metrics from previous literature in PD 4 , 116 , 117 , and (2) using the new Javascript toolbox audio_tokens 74 to collect speech impairment ratings from non-experts. The automated extraction approach quantified the following features of the speech data using Praat 118 with the Python-parselmouth interface 119 : harmonics-to-noise ratio (hnr), timing and amplitude fluctuations of glottal pulses (jitter and shimmer, respectively), standard deviation of pitch measured over voiced segments (f0_std), and a proxy for vowel space (area), namely the product of the inter-quartile range of F1 and F2 values measured over voiced segments (as a simplified version of the procedure described in Sandoval et al 116 ).…”

“…"), and prosody (instruction: "Does the speech sound flat or monotonous?"). Sample ratings were performed in the audio_tokens toolbox 74 (Fig. 1A), which allowed for interactive and dynamic comparison of speech samples across patients.…”

Moving Experimental Psychology Online: How to Maintain Data Quality When We Can’t See Our Participants

“…The push towards moving our experiments online has also resulted in a flurry of new research opportunities. The year 2022 alone saw the publication of tools to collect eye-tracking data from infants in their homes using an online webcam-linked eye tracker (OWLET; Werchan et al, 2022), adaptive methods for estimating psychometric function parameters in online experiments (jsQuestPlus; Kuroki & Pronk, 2022), methods for collecting speeded overt pronunciation responses in language production tasks (Vogt et al, 2022), mouse-tracking methods that provide an alternative eye-tracking in order to track participants' spatial attention (MouseView.js;Anwyl-Irvine et al, 2022), tools to support highly interactive coalition formation experiments (The Online Coalition Game; Wissink et al, 2022), methods to allow researchers using auditory studies to standardize sound level adjustments, detect stereo/mono playback, and assess lower frequency limits (Headphone and Loudspeaker Test (HALT); Wycisk et al, 2022), a toolbox for facilitating the collection of a range of ratings data for auditory material (Donhauser & Klein, 2022), methods for measuring sensorimotor synchronization (SMS) in online experiments using the built-in microphone and speakers of standard laptop computers (Anglada-Tort et al, 2022).…”