Accent rating by native and non-native listeners

Wester, Mirjam; Mayo, Cassie

doi:10.1109/icassp.2014.6855098

Cited by 5 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, non-native teachers rated fluency lower than did the native raters (Gilquin et al 2016;Rose 2017). Some studies also showed that native speakers seemed to be more tolerant than non-native speakers when judging accent (Koster and Koet 1993;Wester and Mayo 2014). Winke et al (2012) also found that raters whose L2 is the test takers' L1 tended to be lenient in scoring.…”

Section: Assessing Spoken Learner English Using Cefrmentioning

confidence: 99%

Evaluating CEFR rater performance through the analysis of spoken learner corpora

Huang

Kubelec²,

Keng

et al. 2018

Lang Test Asia

View full text Add to dashboard Cite

Background: Although teachers of English are required to assess students' speaking proficiency in the Common European Framework of Reference for Languages (CEFR), their ability to rate is seldom evaluated. The application of descriptors in the assessment of English speaking on CEFR in the context of English as a foreign language has not often been investigated, either. Methods: The present study first introduced a form of rater standardization training. Two trained raters then assessed the speaking proficiency of 100 learners by means of actual corpus data. The study then compared their rating results to evaluate interrater reliability. Next, ten samples of exact/adjacent agreement between Raters 1 and 2 were rated by six teachers of English in tertiary education. Two of them had attended rater standardization training with Raters 1 and 2, while the other four had not received any relevant training. Results: The two raters agreed exactly in 44% of cases. The rating results between the two trained raters were closely correlated (ρ = .893). Cross-tabulation showed that in one third of the samples, Rater 2 scored higher than Rater 1 and they agreed more often at the higher levels. The better rating performance of Teachers 1 and 2 suggested that rater standardization training may have helped enhance their performance. The unsatisfactory proportion of correctly assigned levels in teachers' ratings overall was probably due to the high input of subjective judgment based on vague CEFR descriptors. Conclusions: Regarding assessment, it is shown that the attendance of rater standardization training is of help in assessing learners' speaking proficiency in CEFR. This study provides a model for assessing data from spoken learner corpora, which adds an important attribute to future studies of learner corpora. The paper also raises doubts about teachers' ability to evaluate students' speaking proficiency in CEFR. As CEFR has been widely adopted in the relevant fields of English language teaching and assessment, it is suggested that the rating training framework established in this study, which uses learner corpus data, be offered to (prospective) teachers of English in tertiary education.

show abstract

Section: Assessing Spoken Learner English Using Cefrmentioning

confidence: 99%

Evaluating CEFR rater performance through the analysis of spoken learner corpora

Huang

Kubelec²,

Keng

et al. 2018

Lang Test Asia

View full text Add to dashboard Cite

show abstract

“…The study consists of English speech from native and non-native speakers. We used the same three English utterances used in [15,16,23] spoken by all the speakers for which subjective accentedness ratings are available.…”

Section: Speakersmentioning

confidence: 99%

Automatic accentedness evaluation of non-native speech using phonetic and sub-phonetic posterior probabilities

Rasipuram

Cerňak

Nachen³

et al. 2015

Interspeech 2015

View full text Add to dashboard Cite

Automatic evaluation of non-native speech accentedness has potential implications for not only language learning and accent identification systems but also for speaker and speech recognition systems. From the perspective of speech production, the two primary factors influencing the accentedness are the phonetic and prosodic structure. In this paper, we propose an approach for automatic accentedness evaluation based on comparison of instances of native and non-native speakers at the acousticphonetic level. Specifically, the proposed approach measures accentedness by comparing phone class conditional probability sequences corresponding to the instances of native and non-native speakers, respectively. We evaluate the proposed approach on the EMIME bilingual and EMIME Mandarin bilingual corpora, which contains English speech from native English speakers and various non-native English speakers, namely Finnish, German and Mandarin. We also investigate the influence of the granularity of the phonetic unit representation on the performance of the proposed accentedness measure. Our results indicate that the accentedness ratings by the proposed approach correlate consistently with the human ratings of accentedness. In addition, our studies show that the granularity of the phonetic unit representation that yields the best correlation with the human accentedness ratings varies with respect to the native language of the non-native speakers.

show abstract

“…It may well be that these judges are biased against nonstandard speakers, not only when they listen to their native language speech but also when listening to their target language speech, in which they recognize features of the non-standard language. Indeed, some non-native speaker judges are unable to identify native speakers (Wester & Mayo 2014; anecdotally, we have personally experienced situations where teachers of English who were native speakers of Dutch did not recognize native speakers of English amongst their students, often because these students were speakers of a non-standard variety of English, e.g. from the Windward Islands).…”

Section: Introductionmentioning

confidence: 99%

Pride and prejudice? Judging non-native pronunciation of English

Koet¹,

Bergh²

2018

L1 Educational Studies in Language and Literature

View full text Add to dashboard Cite

Can non-native judges of language proficiency be trusted? To answer this question, we asked Dutch (N = 70) and English (N = 57) listeners to judge English speech produced by Dutch speakers, and we asked Dutch listeners to judge Dutch speech produced by these speakers. Results showed that the Dutch listeners to the English speech produced by the Dutch speakers are stricter about pronunciation than the English listeners; they also arrive at rankings that are different from the rankings arrived at by the English listeners. We have found that the Dutch listeners, when judging the aesthetic quality and intonation of Dutch speakers' pronunciation of English, are influenced by aspects of the speakers' native language which interfere with the target language. Their judgements of English pronunciation may therefore be affected by their impressions of the speakers' Dutch.

show abstract

Accent rating by native and non-native listeners

Cited by 5 publications

References 22 publications

Evaluating CEFR rater performance through the analysis of spoken learner corpora

Evaluating CEFR rater performance through the analysis of spoken learner corpora

Automatic accentedness evaluation of non-native speech using phonetic and sub-phonetic posterior probabilities

Pride and prejudice? Judging non-native pronunciation of English

Contact Info

Product

Resources

About