Validating a rubric for assessing integrated writing in an EAP context

Uludag, Pakize; McDonough, Kim

doi:10.1016/j.asw.2022.100609

Cited by 15 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers focusing on the standardized assessment of L2 writing performance have defined the construct of integrated writing tasks (e.g., Cumming et al, 2006; Knoch & Sitajalabhorn, 2013) and discussed the factors that impact student performance on such tasks (e.g., Gebril & Plakans, 2013; Uludag & McDonough, 2022). However, evaluation of constructs underlying source integration in educational assessment contexts has not gained much research attention.…”

Section: Defining Source Integrationmentioning

confidence: 99%

See 1 more Smart Citation

ExploringEAPinstructors' evaluation of classroom‐based integrated essays

Uludag¹,

McDonough²

2022

TESOL Journal

Self Cite

View full text Add to dashboard Cite

Due to its authenticity as an academic writing task, integrated writing assessment has become widely used for assessing the writing ability of English for academic purposes (EAP) students (Plakans & Gebril, 2017). However, apart from validation studies focusing on a common set of standardized test rubrics (e.g., Chan, Inoue, & Taylor, 2015), little research has explored the construct of integration or how to assess it effectively in EAP contexts. To provide second language (L2) writing researchers and practitioners with an empirically based model of source integration, this study explored EAP instructors' orientations when assessing integrated essays and investigated the relationship between instructor ratings and textual measures of source use. Six experienced EAP instructors first evaluated two sample argumentative essays written by undergraduate English L2 students and provided comments through a stimulated recall interview. Next, they evaluated 48 additional argumentative essays using an analytic rubric that included dimensions of source use and provided comments for each essay. Finally, the essays were analyzed for linguistic and rhetorical features associated with source integration. Triangulation of data sources revealed that the instructors oriented to three aspects of source integration: using source information to support personal claims, incorporating source‐text language in students' own words, and representing source content accurately. Their ratings were correlated with text‐based measures of students' source use. Implications for teaching and assessing integrated writing in EAP settings are discussed.

show abstract

Section: Defining Source Integrationmentioning

confidence: 99%

“…How to cite this article:Uludag, P. & McDonough, K. (2022). Exploring EAP instructors' evaluation of classroom-based integrated essays.…”

mentioning

confidence: 99%

ExploringEAPinstructors' evaluation of classroom‐based integrated essays

Uludag¹,

McDonough²

2022

TESOL Journal

Self Cite

View full text Add to dashboard Cite

show abstract

“…The contested nature of writing and the diverse understandings of writing quality among teachers raise questions about the validity of using these criteria as a measure of writing (Tierney & Simon, 2004; Uludag & McDonough, 2022). Teachers expressed the view that the criteria did not entirely capture all relevant aspects of writing and were thus inadequate for capturing writing achievement.…”

Section: Discussionmentioning

confidence: 99%

“…Primary school teachers in England are responsible for the teaching of all National Curriculum subjects, however, not all will be subject specialists in English. Whether writing assessment criteria are appropriate representations of writing achievement, how teachers interpret these criteria and the consistency of their interpretations are important issues for the assessment of writing (Uludag & McDonough, 2022), as they provide evidence for the validity and reliability of the assessment (Tierney & Simon, 2004). This is especially important for a high-stakes assessment.…”

Section: Introductionmentioning

confidence: 99%

“…Criterion‐referenced assessment is seen as more reliable and less subjective (Gipps, 1994; White, 1994). The validation of assessment criteria can be supported by user perceptions and alignment with the knowledge being measured (Gipps, 1994; Uludag & McDonough, 2022). Assessment criteria that teachers are expected to use typically require a degree of subjective interpretation; however, detailed criteria have been shown to help teachers assess pupils with a greater degree of reliability (Frederiksen & White, 2004; Meisels et al, 2001; Rowe & Hill, 1996; Van der Schaaf et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

‘It's missing the heart of what writing is about’: teachers' interpretations of writing assessment criteria

Clarkson

2023

British Educational Res J

View full text Add to dashboard Cite

Primary school teachers in England are responsible for the statutory assessment of writing, using a set of criteria. The aim of this research was to analyse the responses of teachers to the criteria at the expected standard. An elicitation exercise with 10 primary school teachers was conducted, where teachers talked through their interpretations of the criteria. Qualitative, inductive coding was undertaken to provide a rich and deep understanding of teachers' understandings of the assessment criteria. The perceptions of teachers are shown to be varied and complex. Definitions of the criteria were diverse, and fundamentally different concepts of writing were identified, which points to sources of misunderstanding. Limitations of the assessment criteria themselves are also highlighted, which caused disagreement among the teachers. The findings show evidence of threats to the validity and reliability of the assessment and its use as an accountability measure because of the lack of confidence in inferences drawn from the use of the criteria in the assessment of writing.

show abstract