Measures of Effective Teaching in Developing Countries

Abstract

There is growing interest in observing teachers' classroom practice, for formative feedback to teachers, teacher performance evaluation and incentives, program impact evaluation, and research on the determinants of student learning. For observations to generate meaningful data, they must be carried out using valid instruments, standardized protocols, and trained observers. A number of different observation instruments have been used in the US, but only two have gained any traction to date in developing countries - the Classroom Assessment Scoring System, CLASS, and the Stallings classroom observation instrument. Although both have been validated in developing countries in connection with student learning outcomes, they have never been used in the same setting. This paper exploits an ongoing research program that used the CLASS instrument to assess 7 th grade mathematics classrooms in Chile. In this research we re--code the same teacher videos using the Stallings instrument. We generate the first global evidence on the comparability and validity of the dimensions of teacher practice they measure. Because CLASS is significantly more expensive and complex to use - its training materials are proprietary and observers require a high degree of skillthere is a policy and research interest in analyzing the degree to which the simpler, open--source, Stallings instrument produces similar results.We ranked the Chilean teachers on their scores in each of the CLASS instrument's three domains (classroom organization; emotional support and instructional support) and analyzed the resulting quartile distributions' correlations with key Stallings measures of teacher performance. We found that teachers' scores on the Classroom organization domain of CLASS were modestly correlated with the most important Stallings measures, in all cases in the directions expected and with statistical significance. Teachers' CLASS scores on classroom organization were positively correlated with their use of time for instruction (r=0.340, p=0.015) and negatively correlated with time spent on classroom management (r=--0.321, p=0.022). Teachers in the bottom quartile of the distribution averaged only 80% of time on instruction, while those in the top quartile averaged 90%. Time on classroom management in the top quartile was a very low 9% of class time, while the lowest quartile spent 17%. All differences were statistically significant at the 5% or 1% level.Teachers' scores on the CLASS classroom organization domain were also positively correlated with the Stallings measures of teachers' ability to keep students engaged. However, Stallings measures were not correlated with teachers' scores on the other two CLASS domains: instructional support and

Methodological lessons on measuring quality teaching in Southern contexts, with a focus on India and Pakistan

“…Bruns et al (2016), comparing CLASS with another widely-used observation instrument (Stallings) find that when using them within the same developing country context (Chile), both instruments produce consistent assessments of teachers’ effectiveness in managing their classrooms and ensuring student learning.…”

“…In summary, an increasing body of global evidence on teacher effectiveness has found that teachers are differentially effective in producing student achievement; what they do and how they do it is of crucial importance for student outcomes. However, this cannot be predicted by teachers’ formal qualifications or other aspects of observable teacher characteristics (Bruns et al, 2016). By contrast, the contractual arrangements for teachers and measures of teacher effort do predict student achievement in some contexts.…”

Failing to notice? Uneven teachers’ attention to boys and girls in the classroom

“…The literature on classroom interactions focuses on the nature, quality, and quantity of interactions between teachers and students in different domains in the classroom (Pianta et al 2008a;Stallings and Knight 2003;Bruns et al 2016). CLASS does not examine the content of what is being taught or the specific activities carried out.…”

“…A different strand of the literature examines what happens in the classroom. It generates quantitative data on teachers' practice by coding segments of time and observing activities and participation (Bruns et al 2016). The coding strategy and the variables generated for this study are more closely related to this line of research.…”

Long‐run effects of teachers in developing countries

“…The project found that the best composite measure is based primarily on student test scores (value‐added), with a small weight for lesson observation and student feedback (Kane, McCaffrey, Miller, & Staiger, 2013). Lesson observations might, however, also help to understand why some teachers are more effective than others, by identifying specific practices that are associated with improved outcomes (see, for example, Araujo, Carneiro, Cruz‐Aguayo, & Schady, 2016; Bruns, Gregorio, & Taut, 2016; Carnoy, Ngware, & Oketch, 2015; Ngware, Ciera, Musyoka, & Oketch, 2015).…”