Exploring gender and oral proficiency interview performance

O’Sullivan, Barry

doi:10.1016/s0346-251x(00)00018-x

Cited by 54 publications

(31 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The findings of the study are in accordance with the literature which suggests that the constructirrelevant factors can influence the assessment of the raters and the scores of the test-takers in oral interviews (e.g., Chalhoub-Deville & Wigglesworth, 2005;Myford & Wolfe, 2000;O'Loughlin, 2002;O'Sullivan, 2000;Winke & Gass, 2012;Winke et al, 2011). Several factors that affect raters' scorings in oral interviews have been studied in the literature; however, to the knowledge of the researchers, no study has been conducted to investigate the effects of the raters' prior knowledge of the students' proficiency levels on their scoring behaviors during proficiency exams oral interviews.…”

Section: Resultssupporting

confidence: 89%

“…Previous studies have investigated rater effects on oral test scores from different perspectives such as the raters' educational and professional experience (e.g., Chalhoub-Deville, 1995), raters' nationality and native language (e.g., Chalhoub-Deville & Wigglesworth, 2005;Winke & Gass, 2012;Winke et al, 2011), rater training (e.g., Lumley & McNamara, 1995;Myford & Wolfe, 2000), and the gender of candidates and/or interviewers (e.g., O'Loughlin, 2002;O'Sullivan, 2000). For instance, Lumley and McNamara (1995) examined the effect of rater training on the stability of rater characteristics and rater bias whereas MacIntyre, Noels, and Clément (1997) examined bias in self-ratings in terms of participants' perceived competence in an L2 in relation to their actual competence and language anxiety.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Raters Knowledge of Students Proficiency Levels as a Source of Measurement Error in Oral Assessments

Tanriverdi-Köksal¹

2017

HUJE

View full text Add to dashboard Cite

There has been an ongoing debate on the reliability of oral exam scores due to the existence of human raters and the factors that might account for differences in their scorings. This quasi-experimental study investigated the possible effect(s) of the raters' prior knowledge of students' proficiency levels on rater scorings in oral interview assessments. The study was carried out in a pre-and post-test design with 15 EFL instructors who performed as raters in oral assessments at a Turkish state university. In both pre-and post-tests, the raters assigned scores to the same video-recorded oral interview performances of 12 students from three different proficiency levels. While rating the performances, the raters also provided verbal reports about their thought processes. The raters were not informed about the students' proficiency levels in the pre-test, while this information was provided in the posttest. According to the findings, majority of the Total Scores ranked lower or higher in the post-test. The thematic analysis of the raters' video recorded verbal reports revealed that most of the raters referred to the proficiency levels of the students while assigning scores in the post-test. The findings of the study suggest that besides factors such as accent, nationality, and gender of the test-takers and the assessors, raters' prior knowledge of students' proficiency levels could be a variable that needs to be controlled for more reliable test results.Keywords: rater effects, intra-rater reliability, paired oral exams, think-aloud protocols ÖZ: Yaygın olarak kullanılmakta olsa da notlandıran olarak insan faktörünün varlığı ve notlardaki farklılığa neden olan etmenler sebebiyle konuşma sınav notlarının güvenirliği konusunda süregelen bir tartışma vardır. Bu yarı deneysel çalışma, konuşma sınavlarının değerlendirilmesinde, not verenlerin öğrencilerin dil yeterlilik seviyelerini önceden biliyor olmasının verdikleri notları üzerindeki etkilerini araştırmayı amaçlamaktadır. Bu çalışma, Türkiye'deki bir devlet üniversitesinde yabancı dil olarak İngilizce öğreten ve aynı üniversitede konuşma sınavlarında notlandıran olarak görev alan 15 okutman ile ön ve son test olarak iki oturumda yürütülmüştür. Hem ön hem de son testte, notlandıranlar üç farklı seviyeden 12 öğrencinin video kaydına alınmış aynı konuşma sınavı performansları için not vermiştir. Aynı zamanda, performanslar için not verirken, notlandıranlar eş zamanlı olarak ne düşündükleri ile ilgili sözlü bildirimde bulunmuştur. Öğrencilerin dil yeterlilik seviyeleri ile ilgili ön testte herhangi bir bilgi verilmezken, notlandıranlar öğrencilerin seviyeleri konusunda son testte sözlü ve yazılı olarak bilgilendirilmiştir. Sonuçlara göre, önteste kıyasla son testte Toplam Notların büyük çoğunluğunun son testte düştüğü veya yükseldiği saptanmıştır. Tüm notlandıranların video kayıtlı sözlü bildirimleri tematik olarak incelendiğinde, notlandıranların çoğunun son testte not verirken öğrencilerin dil yeterlilik seviyelerine değindikleri gözlemlenmiştir. Çalışmanın ...

show abstract

Section: Resultssupporting

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Raters Knowledge of Students Proficiency Levels as a Source of Measurement Error in Oral Assessments

Tanriverdi-Köksal¹

2017

HUJE

View full text Add to dashboard Cite

show abstract

“…Thus, many testtakers described themselves as shy or introverted and pointed out they felt more relaxed before a computer without the presence of an examiner. This finding is consistent with previous research which suggests that candidates' personal characteristics such as level of extroversion or introversion can affect their test scores in oral assessments (Kang, 2008;Nakatsuhara, 2010;O'Sullivan, 2000;Ockey, 2009;Underhill, 1987). Other test-takers were also positive about the use of computerised tests since this type of assessment was thought to increase rater reliability.…”

Section: Differences Between the Computer-based Test (Aptis Test) Andsupporting

confidence: 92%

Analysing Test-Takers’ Views on a Computer-Based Speaking Test

Amengual-Pizarro

García-Laborda

2017

Profile: Issues Teach. Prof. Dev.

View full text Add to dashboard Cite

Key words:Computer-based language testing, oral assessment, second language testing.Este estudio analiza la opinión de los candidatos sobre un examen oral con ordenador para averiguar los aspectos que consideran más relevantes en la evaluación oral a través de las nuevas tecnologías y explorar las principales ventajas y desventajas de este tipo de pruebas comparadas con pruebas orales con evaluadores humanos. Se distribuyó un pequeño cuestionario a 80 candidatos que realizaron el examen oral aptis en la Universidad de Alcalá en abril de 2016. Los resultados revelan que los candidatos consideran que las pruebas orales con ordenador son válidas y adecuadas para la evaluación de la competencia oral. Curiosamente, los datos demuestran que las características personales de los candidatos juegan un papel primordial en la elección del método de evaluación más idóneo.Palabras clave: evaluación de lenguas a través del ordenador, evaluación de la destreza oral, evaluación de segundas lenguas.

show abstract

“…Like the present study, most of these studies showed that there was a relationship between gender and test scores, but the findings differed from one study to another one. In other words, some studies reported that students obtained higher scores when interviewed by a male interviewer (Locke, 1984;Porter, 1991a;1991b) and some others found that students got higher scores when interviewed by a female interviewer (Porter and Shen, 1991;O'Sullivan, 2000). Buckigham (1997) is among the researchers who claimed that women might feel more comfortable talking to a female interviewer; she found the same-sex effect on Japanese EFL learners' oral performance.…”

Section: A Interviewers' Gender and Oral Performancementioning

confidence: 99%

“…One of the variables that may have a relationship with test takers' performance in oral exams is the gender of the person with whom they interact (O'Sullivan, 2000). Interviewers usually differ from each other based on their test behavior like, how much support they give to interviewees, how friendly they behave towards interviewees, and to what extent they act their role following relevant instructions (O'Loughlin, 2002).…”

Section: Introductionmentioning

confidence: 99%

Interviewers’ Gender and Interview Topic in Oral Exams

Gholami¹,

Sadeghi²,

Nozad³

2011

TPLS

View full text Add to dashboard Cite

Abstract-Nowadays, language schools tend to include more of oral exams as part of their course completion assessment instruments in addition to traditional written exams. The major purpose of this study was accordingly to investigate the relationship between interviewers' gender, interview topic and oral performance. Thirty female upper-intermediate students were selected. All the participants were interviewed twice, once by a female and once by a male interviewer and each time they were required to talk about two topics, one gender-neutral and another gender-biased. All the performances were scored twice with an inter-rater reliability of +.88 and +.89 for two interviews respectively. A comparative analysis of the interview scores indicated that students achieved higher scores when interviewed by a female interviewer. With respect to interview topic, students performed better on gender-neutral topic.

show abstract

Exploring gender and oral proficiency interview performance

Cited by 54 publications

References 10 publications

Raters Knowledge of Students Proficiency Levels as a Source of Measurement Error in Oral Assessments

Raters Knowledge of Students Proficiency Levels as a Source of Measurement Error in Oral Assessments

Analysing Test-Takers’ Views on a Computer-Based Speaking Test

Interviewers’ Gender and Interview Topic in Oral Exams

Contact Info

Product

Resources

About