Towards Defining a Valid Assessment Criterion of Pronunciation Proficiency in Non-Native English-Speaking Graduate Students

Abstract

Intelligibility has been widely regarded as an appropriate goal for second language pronunciation teaching. Yet there is no universally accepted definition of intelligibility, nor any field-wide consensus on the best way to measure it. Further, there is little empirical evidence to suggest which pronunciation features are crucial for intelligibility to guide teachers in their instructional choices. This mixed-methods study examines whether intelligibility is an appropriate criterion for assessing pronunciation proficiency in the academic domain. Speech samples of eight non-native English speaking graduate students were elicited using the Test of Spoken English, a standardized test often used to screen international teaching assistants (ITAs). Results of a fine-grained analysis of the speech samples combined with intelligibility ratings of 18 undergraduate science students suggest that intelligibility, though an adequate assessment criterion, is a necessary but not a sufficient condition for international graduate students to instruct undergraduate courses as teaching assistants.

Linguistic correlates of comprehensibility in second language Japanese speech

“…Whereas comprehensibility is generally assessed via native speakers' scalar judgement (e.g., Trofimovich & Isaacs, 2012), intelligibility, defined as an actual product of understanding (Derwing & Munro, 2015), has been evaluated by way of a range of outcome measures, such as native speakers' transcriptions of speech samples (Derwing & Munro, 1997), comprehension questions (Hahn, 2004), and impressionistic judgments (Anderson-Hsieh, Johnson, & Koehler, 1992). As Isaacs (2008) argues, the methodological variability of intelligibility measures points to the strong call for more empirical studies which elaborate and validate adequate tasks and methods of analysis specific to various research contexts and goals. Thus, future studies should elaborate and validate reliable assessments to examine the intricate role of pronunciation, fluency and lexicogrammar in determining native speakers' intelligibility judgements.…”

How Well do General-Skills ESL Textbooks Address Pronunciation?

“…Most researchers agree that intelligibility and comprehensibility should be the primary goals of pronunciation instruction, not the eradication of accent (Derwing & Munro, 2005;Isaacs, 2008;Kennedy & Trofimovich, 2008;Levis, 2005). In recent years, investigations have been conducted to assess how various aspects of pronunciation make contributions to intelligibility.…”

“…Although raters will almost always judge unintelligible L2 speech as both incomprehensible and highly accented, the reverse is not necessarily true (Munro & Derwing, 1995a). That is, raters will assess some individuals with strong accents as easy to understand and will find some heavily accented speech fully intelligible.Most researchers agree that intelligibility and comprehensibility should be the primary goals of pronunciation instruction, not the eradication of accent (Derwing & Munro, 2005;Isaacs, 2008;Kennedy & Trofimovich, 2008;Levis, 2005). In recent years, investigations have been conducted to assess how various aspects of pronunciation make contributions to intelligibility.…”

International teaching assistants at universities: A research agenda

“…While ITA educators remain interested in all aspects of pronunciation (Meyers & Holt 2001;Gorsuch 2012a), suprasegmental aspects of pronunciation have gotten some focused attention, which has been expressed in research and materials (see also Isaacs (2008) studies based on ITA learner populations on sentence-level stress (Hahn 2004;Levis, Muller Levis & Slater 2012), and tone choices and intonation (Wennerstrom 1998;Pickering 2001) with an aim to linking ITAs' spoken intelligibility to these aspects of pronunciation. There has been pedagogical intervention research on ITAs' pausing patterns (pauses violating clause and sentence boundaries) (Gorsuch 2011a;) motivated by descriptive studies linking pausing patterns to listener perceptions of spoken fluency (e.g., Ejzenberg 2000;Olynack, Anglejan & Sankoff 1990;Pawley & Syder 2000).…”