Reliability of Speech Variables and Speech-Related Quality Indicators in the Swedish Cleft Lip and Palate Registry

Abstract

Objective: To assess the reliability of speech variables and speech-related quality indicators in the Swedish quality registry for cleft lip and palate (CLP). Design: Retrospective study. Setting: Primary care university hospitals. Participants: Fifty-two 5-year-old children with unilateral CLP and 41 with bilateral CLP. Main Outcome Measures: Registry data for “percent nonoral errors” and “perceived velopharyngeal competence” (VPC) were compared to reassessments by 4 independent judges based on audio recordings. Interjudge agreement for “percent consonants correct” (PCC) and the reliability of 3 quality indicators were also assessed. Agreement was calculated with single measures intraclass correlation coefficient (ICC) for articulation outcomes, quadratic weighted κ and ICC for VPC, and percentage agreement and κ for quality indicators. Results: When the agreement between registry data and the judges’ reassessments was assessed, the ICC was 0.79 for percent nonoral errors. For VPC, the κ coefficient was 0.66 to 0.75 and the ICC was 0.73. Interjudge agreement for PCC calculated with ICC was 0.85. For the quality indicator “proportion of children with ≥86% correct consonants,” all 4 judges were in agreement for 72% of the cases. For “proportion of children without nonoral speech errors” and “proportion of children with competent or marginally incompetent velopharyngeal function,” the agreement between registry data and the 4 judges was 89% and 85%, respectively. Conclusions: The results indicate that registry data on PCC, percent nonoral errors, VPC, and the quality indicators “proportion of children without nonoral speech errors” and “proportion of children with competent or marginally incompetent velopharyngeal function” are reliable.

Coverage, reporting degree and design of the Swedish quality registry for patients born with cleft lip and/or palate

“…Percentage measures of consonant production have been widely used in cleft palate speech research [27]. In the study by Brunnegård et al [29], agreement between judges for the percentage of correct consonants was excellent, with a singlemeasures intra-class coefficient of 0.85. Further, reliability of data on the percentage of non-oral errors in the CL/P registry was excellent in the study by Brunnegård et al, and in a previous study [31].…”

“…Perceived velopharyngeal competence, i.e., an overall assessment of hypernasality, audible nasal air leakage and weak articulation, is rated on a three-point scale with the scale values 'competent/ sufficient', 'marginally incompetent/insufficient' and 'incompetent/insufficient' [27]. Perceived velopharyngeal competence has previously been validated [28], and in a study by Brunnegård et al [29], the reliability of data on perceived velopharyngeal competence in the CLP registry was good to excellent. The percentage of consonants correct and percentage of non-oral speech errors are recorded [27], based on phonetic transcriptions according to the International Phonetic Alphabet [30], for the 59 target consonants in SVANTE [27].…”

“…Since the reporting degree and reliability of speech data in the Swedish CLP registry has proven to be good, speech data allow for open comparisons. Based on the speech variable percentage of non-oral errors included in the CLP registry, Brunnegård et al [29] developed a quality indicator, Proportion of children without non-oral errors. To have a margin of error, children were allowed to have up to 5% non-oral errors without being counted as having non-oral errors, and this quality indicator was found to be reliable.…”

“…These two quality indicators have been published for open comparison of data from the CLP registry online [10]. The third quality indicator, Proportion of children with > 86% correct consonants, needs further evaluation before publication on the Internet [29].…”

Inter-centre comparison of data on surgery and speech outcomes at 5 years of age based on the Swedish quality registry for patients born with cleft palate with or without cleft lip

“…Several studies have assessed the reliability of speech data in the Swedish CLP registry [ 21 – 23 ]. Brunnegård et al [ 22 ] assessed the reliability of data on perceived velopharyngeal competence in the CLP registry, which was good to excellent. Furthermore, data on percentage consonants correct and percentage non-oral speech errors have been proven to be satisfying [ 22 , 23 ].…”