Automated Text Analytics in Swedish English Classrooms: The Design, Development, and Evaluation of a Supporting Tool

Abstract

This dissertation explores the design and application of a digital tool as a supporting aid for teachers of English in Swedish upper-secondary classrooms. The scope of this dissertation is the design, development and evaluation of a focus-on-form diagnostics tool in the Swedish upper-secondary English subject context. The use of corrective feedback on student texts has been discussed extensively in previous research, but the implementation of an automated diagnostics tool as a support for the practice less so. The teachers' needs are explored through a survey with the goal of gaining insight into problems learners commonly have when attaining grammar, vocabulary and spelling as experienced by in-service teachers. Focus is on how a digital diagnostic tool could fit within the teachers’ current routines. By combining the survey results with a review of previous research, a design specification for the tool is produced. The tool is then evaluated in teaching practice as a way of exploring the possible benefits and issues of implementing automated language diagnostics as a support for teachers. The evaluation data was produced through interventions and debriefing interviews with the participating teachers. The results of the survey on teachers' needs indicated that any automated attempt at supporting language diagnostics within the school context would have to be able to identify a series of error patterns, such as sentence structure, punctuation, verb tenses, subject-verb agreement, article usage and spelling. In addition, the tool would have to fit within an already established practice aimed at providing learners with the necessary supports for them to succeed, while also avoiding pitfalls such as creating avoidance issues or overwhelming the students. The results of the intervention study used to evaluate the tool indicated that the respondents found it likely that the implementation of the tool would result in a higher workload initially, but would be beneficial long-term. In terms of fit with current routines and ability to identify relevant language patterns, the tool was experienced as conceptually sound. However, some issues, such as the limited target variant, technical issues surrounding on-site implementation, and the overwhelming form of the output data, remain. The use of automated text analysis for the focus-on-form reading performed by the teachers was found to provide functional support for them. Based on the results, automated diagnostics tools are indicated as being a fitting venue for future research and development.