No abstract

In this research report, we describe the design and empirical findings for a large‐scale study of essay writing ability with approximately 2,500 high school students in Germany and Switzerland on the basis of 2 tasks with 2 associated prompts, each from a standardized writing assessment whose scoring involved both human and automated components. For the human scoring aspect, we describe the methodology for training and monitoring human raters as well as for collecting their ratings within a customized platform. For the automated scoring aspect, we describe the methodology for training, evaluating, and selecting appropriate automated scoring models as well as correlational patterns of resulting task scores with scores from secondary measures. Analyses show that the human ratings were highly reliable and that effective prompt‐specific automated scoring models could be built with state‐of‐the‐art features and machine learning methods, which resulted in correlational patterns with secondary measures that were in line with general expectations. In closing, we discuss the methodological implications for conducting this kind of work at scale in the future.

No abstract

Zusammenfassung Produktive Sprachkompetenzen im Fach Englisch sind bislang in Deutschland nur wenig untersucht worden. Daher wurden in der vorliegenden Untersuchung mit zwei Messzeitpunkten Kompetenzen im argumentativen und sachorientierten Schreiben von N = 838 Schülerinnen und Schülern in der 11. Jahrgangsstufe Die Daten der vorliegenden Studie stammen aus dem Projekt "Measuring English Writing at Secondary Level (MEWS)", das von der Deutschen Forschungsgemeinschaft (DFG; GZ: KO1513/12-1) und vom Schweizerischen National Fonds (SNF: Fördernr. 100019L162675) gefördert wird. K 1282 O. Köller et al. des achtjährigen Gymnasiums untersucht. Zusätzlich wurden rezeptive Kompetenzen (Hören und Lesen) berücksichtigt. Mit Blick auf die Ziele der gymnasialen Oberstufe (Erreichen des Niveaus B2 des Gemeinsamen Europäischen Referenzrahmens für Sprachen; GER) konnte gezeigt werden, dass bereits ein Jahr vor Erreichen des Abiturs rund 60 % der getesteten Schülerinnen und Schüler das Niveau B2 oder höher im Schreiben erreichten. Weiterhin belegen die Analysen signifikante Kompetenzzuwächse im Laufe eines Schuljahres und zeigen deutliche Unterschiede zwischen den Profilen (sprachlich vs. naturwissenschaftlich vs. gesellschaftswissenschaftlich vs. sonstige), die die Schülerinnen und Schüler in der gymnasialen Oberstufe belegt haben; hier zeigen Jugendliche in den Sprachprofilen signifikant höhere Leistungen. Die Ergebnisse werden mit Bezug auf normative Zielvorgaben der gymnasialen Oberstufe diskutiert. Schlüsselwörter Fremdsprachenkompetenzen · Gymnasiale Oberstufe · Gemeinsamer Europäischer Referenzrahmen für Sprachen · Ziele schulischen Lernens Writing skills in English as a foreign language in upper secondary schoolAbstract There is a substantial lack of studies in productive skills of German students in English as a foreign language. Based on this research gap, the present repeated measurement study evaluated the competence to write argumentative essays and synthesis texts of n = 838 11th graders from upper secondary schools while also collecting data on receptive skills (listening and reading comprehension). Findings show that a substantial number of students (approximately 60%) reach level B2 or higher of the Common European Framework of Reference for Languages one year before leaving upper secondary school, which is the official standard for upper secondary school writing. Furthermore, analyses show that all foreign language skills increased over the course of one school year and that students with different educational emphases in their school tracks (i.e., with a primary focus on languages vs. natural science vs. social science vs. other subjects) differ significantly in their writing skills. Specifically, students who pursue language-centered studies clearly outperformed all other students as expected. Findings are discussed with respect to normative performance expectations of upper secondary schooling.Keywords Common European Framework of Reference for Languages · Foreign language skills · Normative goals of schooli...

No abstract