Exploring the reliability of generic and content-specific instructional aspects in physical education lessons

Charalambous, Charalambos Y.; Kyriakides, Ermis; Tsangaridou, Niki; Kyriakidēs, Leōnidas

doi:10.1080/09243453.2017.1311929

Cited by 10 publications

(6 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In line with Murray (1983) and ample evidence that classroom observation reliability increases substantially when an increasing number of observers visit different lessons (e.g., Charalambous et al 2017;Hill et al 2012;Praetorius et al 2014;van der Lans et al 2016), this study predicts that the association between student survey and classroom observation measures increases with the number of lesson visits by different observers. Formally, & H1 0 : The association between the survey and classroom observation measures decreases or does not increase with an increasing number of classroom observations.…”

Section: Hypothesessupporting

confidence: 66%

“…He also correlated these data with student ratings obtained in the same class. The use of multiple observers visiting multiple lessons increases the reliability of classroom observation measures (e.g., Charalambous et al 2017;Hill et al 2012;Kane et al 2012;Praetorius et al 2014). Moreover, unreliability in measurements also attenuates the correlation between measures (Spearman 1904).…”

Section: The Association Between Survey and Observation Measuresmentioning

confidence: 99%

“…The observer facet suggests that the students' scores are affected by their personal norms of good teaching, to a greater extent than trained observers' are, which is logical given the difference in training. Furthermore, the teacher × observer facet shows more variation in the classroom observation measure than the student survey measure, which again makes sense; scores on classroom observation measures should vary over lessons and teacher-observer combinations (e.g., Charalambous et al 2017;Praetorius et al 2014), whereas scores on student survey measures are expected to be more lesson-generic and vary mostly over teacher-student combinations. The class effects possibly reflect differences in teachability of classes.…”

Section: Modelmentioning

confidence: 99%

See 2 more Smart Citations

On the “association between two things”: the case of student surveys and classroom observations of teaching quality

Lans

2018

Educ Asse Eval Acc

View full text Add to dashboard Cite

This study examines the association between two measures of teaching effectivenessa student survey measure and a classroom observation measure-to determine whether their correlation depends on the study design. The sample includes 160 classroom observations of 56 teachers across 15 classes, in which students also rated the teachers with a survey. Dependencies are examined using generalizability theory. Results suggest that the correlation between the survey and observation measures depends on the number of classroom observations, the number of student ratings, and whether the designs are nested or partially nested. The effect is substantial: Predicted correlations range between 0.10-0.80 for the same variables with different study designs. In particular, the number of classroom observations has a notable influence, such that across all investigated scenarios, the correlation doubles when observers visit three lessons instead of one. Correlations also tend to be positively biased when research designs are nested. Keywords Teacher evaluation. Teacher assessment. Teaching effectiveness. Composite measures. Generalizability theory This study examines the association between two measures of teaching effectivenessa student survey measure and a classroom observation measure-to determine whether the correlation depends on the study design. Educational policies around the globe show increasing interest in measuring high-quality, highly effective teaching (e.g., Isoré 2009; Mourshed et al. 2010; National Council on Teacher Quality [NCTQ], 2013).

show abstract

Section: Hypothesessupporting

confidence: 66%

Section: The Association Between Survey and Observation Measuresmentioning

confidence: 99%

Section: Modelmentioning

confidence: 99%

See 1 more Smart Citation

On the “association between two things”: the case of student surveys and classroom observations of teaching quality

Lans

2018

Educ Asse Eval Acc

View full text Add to dashboard Cite

show abstract

“…Prominent examples of these facets include the choice of rater, lesson, lesson segment for an observation, as well as the mode in which observations are scored. A number of studies have used generalizability theory (G-Theory; Brennan, 2001; Cronbach, Gleser, Harinder Nanda, & Rajaratnam, 1972) to decompose the variance of observation protocol scores into that which is attributable to the true differences between teachers of interest and that which is attributable to the facets most central to the protocol scoring design (Bell et al, 2012; Casabianca, McCaffrey, et al, 2013; Charalambous, Kyriakides, Tsangaridou, & Kyriakides, 2017; Hill, Charalambous, & Kraft, 2012; Ho & Kane, 2013; Mashburn, Meyer, Allen, & Pianta, 2014). However, in all of these studies, it can be argued that there is an important “hidden” facet to the measurement procedure involved in the scoring of a classroom observation.…”

Section: Introductionmentioning

confidence: 99%

Making Inferences About Teacher Observation Scores Over Time

Briggs

Alzen

2019

Educational and Psychological Measurement

View full text Add to dashboard Cite

Observation protocol scores are commonly used as status measures to support inferences about teacher practices. When multiple observations are collected for the same teacher over the course of a year, some portion of a teacher’s score on each occasion may be attributable to the rater, lesson, and the time of year of the observation. All three of these are facets that can threaten the generalizability of teacher scores, but the role of time is easiest to overlook. A generalizability theory framework is used in this study to illustrate the concept of a hidden facet of measurement. When there are many temporally spaced observation occasions, it may be possible to support inferences about the growth in teaching practices over time as an alternative (or complement) to making inferences about status at a single point in time. This study uses longitudinal observation scores from the Measures of Effective Teaching project to estimate the reliability of teacher-level growth parameters for designs that vary in the number and spacing of observation occasions over a 2-year span. On the basis of a subsample of teachers scored using the Danielson Framework for Teaching, we show that at least eight observations over 2 years are needed before it would be possible to make distinctions in growth with a reliability coefficients of .39.

show abstract

“…An additional plausible explanation for these results may be that the content of the 12 lessons focused on passing and movement rather than on the other two technical skills of dribbling and shooting. At the same time, we cannot forget that the factor related to the quality of student practice was identified as the most crucial factor for their learning [65], although it is important to grade the importance of task organization to enable learners to accumulate high-quality repetitions, as this has been found to correlate with improved skill learning. Related to the first point, during learning sessions the frequency of shots on goal was low and unequally distributed among participants.…”

Section: The Effect Of Play-discuss-play On Skill Testsmentioning

confidence: 99%

Effects of Verbal Interactions between Students on Skill Development, Game Performance and Game Involvement in Soccer Learning

et al. 2020

View full text Add to dashboard Cite

The purpose of the study was to examine the effects of verbal interaction between students on skill development and soccer game performance within a socio-constructivist perspective and a cooperative learning model in team-sport teaching. In addition, the usefulness of open verbalization was manifested as follows: (1) a social tool for both actors (teachers and students) to collect and manage reports on their thought processes; (2) a tool to stimulate reflection and critical reflection on performance to induce transformation during game action projects. Participants were 18 boys and 12 girls aged (15 ± 0.4 years) from a Tunisian school (ninth grade). They were placed in either the experimental group (with verbal interaction) or the comparison group (without verbal interaction) and then were tested before and after a 12-lesson soccer unit (approximately two hours/week). Skill competence was assessed using three tests: a 15 m ball dribbling test, the Loughborough Soccer Passing Test (LSPT) and a shooting accuracy test. Game performance was measured using the Game Performance Assessment Instrument (GPAI) in which the outcome variables assessed included (a) decision-making (DM), (b) skill execution (SE), (c) support (S), (d) game performance (GP), and (e) game involvement (GI). While both groups showed significant improvements in their short-passing ability, no such improvements were found in dribbling and shooting. In contrast, only the verbal interaction group produced significant improvements in overall game performance. In conclusion, if the objectives of the physical education curriculum are to promote team-sport teaching methods and quality game play, and create a reflexive learner, verbal interaction may be an effective tool for developing tactical understanding through cooperative learning.

show abstract

Exploring the reliability of generic and content-specific instructional aspects in physical education lessons

Cited by 10 publications

References 53 publications

On the “association between two things”: the case of student surveys and classroom observations of teaching quality

On the “association between two things”: the case of student surveys and classroom observations of teaching quality

Making Inferences About Teacher Observation Scores Over Time

Effects of Verbal Interactions between Students on Skill Development, Game Performance and Game Involvement in Soccer Learning

Contact Info

Product

Resources

About