Absolute and Relative Reliability of Percentage of Syllables Stuttered and Severity Rating Scales

Karimi, Hamid; O’Brian, Sue; Onslow, Mark; Jones, Mark

doi:10.1044/2014_jslhr-s-13-0106

Cited by 20 publications

(15 citation statements)

References 91 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, this measure might be more beneficial in research settings rather than in clinic settings. It should be mentioned that the same issue of low absolute reliability has been also previously reported for other frequently used measures of %SS and Severity Rating (Karimi, O'Brian, Onslow, & Jones, 2014) and it is not just pertinent only to the SCESS.…”

Section: Discussionsupporting

confidence: 54%

The Satisfaction with Communication in Everyday Speaking Situations (SCESS) scale: An overarching outcome measure of treatment effect

Karimi

Onslow

Jones

et al. 2018

Journal of Fluency Disorders

Self Cite

View full text Add to dashboard Cite

Highlights The Consolidated Standards of Reporting Trials (CONSORT) strongly suggests one primary outcome for clinical trials. That presents a roadblock to eventual meta-analysis of randomized trials for adults who stutter. We propose a simple and convenient outcome measure for clinical trials of stuttering treatment that spans behavioral and psychosocial factors that might impel clients to seek treatment  Reliability, content validity and construct validity of the SCESS were confirmed. We conclude that the SCESS measure has potential to be applied as an overarching clinical trial outcome measure of stuttering treatment effect.

show abstract

Section: Discussionsupporting

confidence: 54%

The Satisfaction with Communication in Everyday Speaking Situations (SCESS) scale: An overarching outcome measure of treatment effect

Karimi

Onslow

Jones

et al. 2018

Journal of Fluency Disorders

Self Cite

View full text Add to dashboard Cite

show abstract

“…For absolute reliability, standard error of measurement (SEM) and limits of agreement (LOA) are used for measurement (Batterham & George, 2003; Karimi et al., 2014). Both SEM and LOA can be calculated from the ANOVA in the ICC(2,1) model.…”

Section: Methodsmentioning

confidence: 99%

“…Both SEM and LOA can be calculated from the ANOVA in the ICC(2,1) model. The SEM is an indication of the precision of the measurement (Karimi et al., 2014) and is calculated by:

\begin{equation*} \left(\sqrt{\frac{\mathrm{SST}}{n-1}}\right)\left(\sqrt{1-\mathrm{ICC}}\right) \end{equation*}

where SST is the total sum of squares (SST) in the ANOVA, and n is the number of observations.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

The reliability of simultaneous versus individual data collection during stuttering assessment

Davidow

Edge

2023

Intl J Lang & Comm Disor

View full text Add to dashboard Cite

Background Speech–language pathologists often multitask in order to be efficient with their commonly large caseloads. In stuttering assessment, multitasking often involves collecting multiple measures simultaneously. Aims The present study sought to determine reliability when collecting multiple measures simultaneously versus individually. Methods & Procedures Over two time periods, 50 graduate students viewed videos of four persons who stutter (PWS) and counted the number of stuttered syllables and total number of syllables uttered, and rated speech naturalness. Students were randomly assigned to one of two groups: the simultaneous group, in which all measures were gathered during one viewing; and the individual group, in which one measure was gathered per viewing. Relative and absolute intra‐ and inter‐rater reliability values were calculated for each measure. Outcomes & Results The following results were notable: better intra‐rater relative reliability for the number of stuttered syllables for the individual group (intraclass correlation coefficient (ICC) = 0.839) compared with the simultaneous group (ICC = 0.350), smaller intra‐rater standard error of measurement (SEM) (i.e., better absolute reliability) for the number of stuttered syllables for the individual group (7.40) versus the simultaneous group (15.67), and better inter‐rater absolute reliability for the total number of syllables for the individual group (88.29) compared with the simultaneous group (125.05). Absolute reliability was unacceptable for all measures across both groups. Conclusions & Implications These findings show that judges are likely to be more reliable when identifying stuttered syllables in isolation than when simultaneously collecting them with total syllables spoken and naturalness data. Results are discussed in terms of narrowing the reliability gap between data collection methods for stuttered syllables, improving overall reliability of stuttering measurements, and a procedural change when implementing widely used stuttering assessment protocols. WHAT THIS PAPER ADDS What is already known on the subject The reliability of stuttering judgments has been found to be unacceptable across a number of studies, including those examining the reliability of the most popular stuttering assessment tool, the Stuttering Severity Instrument (4th edition). The SSI‐4, and other assessment applications, involve collecting multiple measures simultaneously. It has been suggested, but not examined, that collecting measures simultaneously, which occurs in the most popular stuttering assessment protocols, may result in substantially inferior reliability when compared to collecting measures individually. What this paper adds to existing knowledge The present study has multiple novel findings. First, relative and absolute intra‐rater reliability were substantially better when stuttered syllables data were collected individually compared to when the same data were collected simultaneously with total number of syllables and speech naturalness dat...

show abstract

“…These are commonly used both in speech therapy [18] and in clinical research [27]. For clinical purpose, severity rating scales are more reliable than % SS , to provide a statement about individual stuttering severity [11].…”

Section: Introductionmentioning

confidence: 99%

Towards Automated Assessment of Stuttering and Stuttering Therapy

Bayerl¹,

Hönig²,

Reister³

et al. 2020

Preprint

View full text Add to dashboard Cite

Stuttering is a complex speech disorder that can be identified by repetitions, prolongations of sounds, syllables or words and blocks while speaking. Severity assessment is usually done by a speech therapist. While attempts at automated assessment were made, it is rarely used in therapy. Common methods for the assessment of stuttering severity include percent stuttered syllables (% SS), the average of the three longest stuttering symptoms during a speech task or the recently introduced Speech Efficiency Score (SES). This paper introduces the Speech Control Index (SCI), a new method to evaluate the severity of stuttering. Unlike SES, it can also be used to assess therapy success for fluency shaping. We evaluate both SES and SCI on a new comprehensively labeled dataset containing stuttered German speech of clients prior to, during and after undergoing stuttering therapy. Phone alignments of an automatic speech recognition system are statistically evaluated in relation to their relative position to labeled stuttering events. The results indicate that phone length distributions differ in respect to their position in and around labeled stuttering events.

show abstract

Absolute and Relative Reliability of Percentage of Syllables Stuttered and Severity Rating Scales

Abstract: Measurement of group changes of stuttering severity can be done in research contexts using either %SS or SR. However, for detecting changes within individuals using such measures, a paired-judge procedure is a more reliable method.

Cited by 20 publications

References 91 publications

The Satisfaction with Communication in Everyday Speaking Situations (SCESS) scale: An overarching outcome measure of treatment effect

The Satisfaction with Communication in Everyday Speaking Situations (SCESS) scale: An overarching outcome measure of treatment effect

The reliability of simultaneous versus individual data collection during stuttering assessment

Towards Automated Assessment of Stuttering and Stuttering Therapy

Contact Info

Product

Resources

About