Exploring Differences Between Chinese and American EFL Teachers' Evaluations of Speech Performance

Gui, Min

doi:10.1080/15434303.2011.614030

Cited by 14 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among studies concerning rater background, the issue of native and non-native speakers (NNS) and their rating patterns have been explored by multiple scholars. Some studies found significant differences based on the L1 background (Zhang and Elder 2014;Gui, 2012) while others found no significant differences (Johnson and Lim's 2009;Barkaoui, 2011). Research that found a minimal effect of L1 language background had NNS raters who possessed near-native language fluency (Johnson and Lim, 2009;Barkaoui, 2011).…”

Section: Different Rater Background Effectsmentioning

confidence: 99%

Writing scale effects on raters: an exploratory study

Jeong¹

2019

Lang Test Asia

View full text Add to dashboard Cite

In writing assessment, finding a valid, reliable, and efficient scale is critical. Appropriate scales, increase rater reliability, and can also save time and money. This exploratory study compared the effects of a binary scale and an analytic scale across teacher raters and expert raters. The purpose of the study is to find out how different scale types impact rating performance and scores. The raters in this study rated twenty short EFL essays using the two scales, completed a rater cognition questionnaire, and took part in an in-depth interview. The ratings were analyzed using a multi-faceted Rasch analysis to compare essay scores and rater statistics across scales and rater groups. The results indicated when using the binary scale, the raters spent less time and were less spread out and more consistent in their ratings. Three out of four raters replied that less mental effort was required when using the binary scale and felt more confident in their ratings. Across the two rater groups, there was a bigger shift in rating performance when using the binary scale for the teacher raters than the expert raters. This implies that scale design had a greater effect on teacher raters. The overall findings suggest that the binary scale maybe a better fit for large scale assessment with sufficient rater training.

show abstract

Section: Different Rater Background Effectsmentioning

confidence: 99%

Writing scale effects on raters: an exploratory study

Jeong¹

2019

Lang Test Asia

View full text Add to dashboard Cite

show abstract

“…However, the results revealed that the rubrics produced little impact on participants' pronunciation, grammar accuracy, or the range of tense usage. Although some common pronunciation errors have been listed in the rubrics, guided by Gui's (2012) report, some students still cannot differentiate the contrasts between /l/-/r/ or /th/-/s/, /r/ sound is mistakenly added to the endings of such words as "China" and "idea", or syllables are incorrectly added or deleted.…”

Section: Resultsmentioning

confidence: 99%

Articulating Teachers’ Expectations afore: Impact of Rubrics on Chinese EFL Learners’ Self-Assessment and Speaking Ability

Huang¹,

Gui²

2014

Journal of Education and Training Studies

Self Cite

View full text Add to dashboard Cite

Under the constraint of classroom time, English as foreign language (EFL) learners usually do not have adequate opportunity to practice oral English in class nor receive individualized feedback from the instructor. After-class practice constitutes an essential component of foreign language learning. This study investigated the effect of utilizing rubrics on EFL learners' oral English using a 16-week experiment and a questionnaire. A total of 61 college students participated in the study, 41 in the rubric-mediated group and 20 in the control group. They were required to complete six tasks of three types: conversation, description, and discussion. Participants in the rubric-mediated group were provided with three rubrics describing respective task assessment criteria. They were also required to record their speaking, self-assess their performance with reference to the rubrics, revise their speaking if they noticed they did not meet the criteria, and finally complete a questionnaire. Students in the control group were assigned the same tasks but were not given the rubrics. Two oral tests were delivered at the initial and final stages of the study to measure the change in their speaking ability. Results revealed that rubrics had positive effects on discourse length, discourse organization, and linguistic flexibility but little impact on linguistic accuracy. Responses to the questionnaire indicated that students perceive rubrics as a facilitative tool of practice and self-assessment.

show abstract

“…In her study a group of teachers of Arabic as a foreign language exerted the heaviest emphasis on creativity in presenting information, while a group of nonteaching Arabs residing in the United States placed the heaviest emphasis on amount of detail provided, and a group of nonteaching Arabs living in their home country exhibited a sole emphasis on grammar-pronunciation. Similarly, Gui (2012) regarded American and Chinese EFL teachers as two distinct groups of raters who differed in their comments on pronunciation, use of English expressions, and speech delivery. In a more recent study Kim (2015) classified her raters into novice, developing, and expert rater groups.…”

Section: Rater Classificationmentioning

confidence: 99%

“…Second, the rating process of each rater type is explored through verbal protocols. While there have been studies on what raters focus on in speaking tests and speech contests (Brown, 2000;Chalhoub-Deville, 1995;Gui, 2012;Kim, 2015;Orr, 2002;Pollit & Murray, 1996), the classification of raters according to their weighting patterns is a relatively new undertaking (Eckes, 2009).…”

Section: Introductionmentioning

confidence: 99%

“…Similarly, Brown (2000) compared the criteria attended to by the International English Language Testing System 2 (IELTS) raters with those specified in the performance descriptors of second language speaking proficiency, and Orr (2002) used verbal report protocols to explore the scoring criteria that raters focused on while rating the Cambridge First Certificate in English 3 (FCE) speaking test. Gui (2012) extended the scope of this work from speaking tests to speech contests when she compared American and Chinese EFL teachers' foci in evaluating student speaking performance. Chalhoub-Deville (1995), however, took a quantitative approach to examining rater weighting.…”

mentioning

confidence: 98%

See 1 more Smart Citation

Weight-Based Classification of Raters and Rater Cognition in an EFL Speaking Test

Cai

2015

Language Assessment Quarterly

View full text Add to dashboard Cite

This study is an attempt to classify raters according to their weighting patterns and explore systematic differences between rater types in the rating process. In the context of an EFL speaking test, 126 raters were classified into three types-form-oriented, balanced, and content-oriented-through cluster analyses of their weighting patterns derived from a holistic judgment task. In subsequent verbal protocols, a smaller sample of raters from each type rated the performance of real test takers and justified their ratings. The procedure for classifying raters yielded reliable results that agreed broadly with self-perceived and actual weights of the raters in the rating process. Verbal protocol analyses found systematic differences across rater types in the distinction between criteria and conformity to rubric-implied criteria. The form-oriented raters were found to overweigh form-related criteria and underweigh content-related criteria, which suggested unbalanced construct representation and led to more halos. Preliminary evidence also suggested that the form-oriented raters might overrate digressive content due to lower sensitivity to digression.

show abstract

Exploring Differences Between Chinese and American EFL Teachers' Evaluations of Speech Performance

Cited by 14 publications

References 14 publications

Writing scale effects on raters: an exploratory study

Writing scale effects on raters: an exploratory study

Articulating Teachers’ Expectations afore: Impact of Rubrics on Chinese EFL Learners’ Self-Assessment and Speaking Ability

Weight-Based Classification of Raters and Rater Cognition in an EFL Speaking Test

Contact Info

Product

Resources

About