Investigating language assessment literacy: Collaboration between assessment specialists and Canadian university admissions officers

Abstract

There are increasing numbers of non-native English speaking applicants to Canadian universities (AUCC 2008a, 2010), which are committed to promoting linguistic and cultural diversity (AUCC 2008b). One result of this trend is that university admissions officers, as gatekeepers, are faced with a growing and potentially confusing array of language test scores when making their decisions. These admissions decision makers need a certain amount of language assessment literacy (LAL) to enable them to make use of these language test scores effectively and ethically (O’Loughlin 2011, 2013). This article reports on the first phase of a project designed to address this challenge. The project involves the collaboration of assessment professionals and admissions officers across Canada in determining the LAL base needed for users of language test scores in university admissions decision-making.This first phase of research consisted of a survey with university admissions officers across Canada, inquiring about their knowledge, beliefs, and levels of confidence in making use of language test scores in decision-making. Results have begun to reveal the nature of the LAL needed for these users, and have suggested the most appropriate content for later informational workshops with admissions officers (Phase 2 of the project). While some evidence of misunderstanding was identified, respondents demonstrate awareness of concepts related to validity in language assessment, albeit without making use of the conventional language of the field.

Examining the assessment literacy required for interpreting score reports: A focus on educators of K–12 English learners

“…Similarly, a number of studies have been conducted to understand university-level ESL educators' or administrators' assessment literacy (e.g., Baker, Tsushima, & Wang, 2014;Coleman, Starfield, & Hagan, 2003;O'Loughlin, 2011O'Loughlin, , 2013. For instance, O'Loughlin (2013) examined how university educators, administrators, and admissions staff interpreted and used International English Language Testing System (IELTS) scores.…”

“…In a similar study, Baker et al (2014) conducted a survey study to examine Canadian university admissions officers' (n = 19) assessment literacy. The findings from this study were more positive than those of other research discussed above, as the participants demonstrated their knowledge of basic concepts related to language assessment, such as cut-off scores for making decisions, and showed a strong interest in developing their assessment literacy.…”

“…Results from both O'Loughlin (2013) and Baker et al (2014) highlighted the importance of assessment literacy related to score reports in the interpretation and communication of scores (Xu & Brown, 2016). In other words, educators need to know how to interpret evidence generated from assessments; they should also know suitable methods for communicating assessment scores to relevant stakeholders, including students, parents, and administrators.…”

The development of EFL examinations in Haiti: Collaboration and language assessment literacy development

“…The majority of empirical work on LAL has investigated language teachers (Brown & Baily, 2008; Inbar-Lourie, 2008; Jeong, 2013; Jin, 2010; Malone, 2013; Vogt & Tsagari, 2014). Other studies have been performed with other users of language test scores such as policy makers (Pill & Harding, 2013), and university admissions officers (Baker, 2016; Baker, Tsushima, & Wang, 2014; Ginther & Elder, 2013; Hyatt & Brooks, 2009; O’Loughlin, 2011, 2013; Rea-Dickins, Kiely, & Yu, 2007; Smith 7 Haslet, 2007). To the authors’ knowledge, there have been no studies that focus on the LAL of language testing professionals.…”

Language assessment literacy in university admission policies, or the dialogue that isn’t

“…The primary focus of LAL research has been on teachers (Fulcher, 2012;Inbar-Lourie, 2017), but considerable attention has also been devoted to law makers (Pill & Harding, 2013) and university admission officers (Baker, 2016;Baker, Tsushima, & Wang, 2014;O'Loughlin, 2011O'Loughlin, , 2013. Most research has focused on the stakeholders who enact policy, but not necessarily on those who construct it (Baker et al, 2014). Nevertheless, understanding the professional context of these policy makers is crucial to an effective collaboration between language assessment professionals and language assessment users.…”