The Reliability of Framework for Teaching Scores in Kindergarten

Patrick, Hugh T.; French, Brian F.; Mantzicopoulos, Panayota

doi:10.1177/0734282920910843

Cited by 8 publications

(11 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our objective with the present study was to address the predictive validity of using FFT scores for teacher evaluation. Our results from kindergarten classrooms suggest that FFT scores capture small, yet measurable, associations between ratings of teachers’ practices and their students’ end-of-year standardized achievement, a finding that is consistent with results from another sample of kindergarten teachers (Patrick, Mantzicopoulos, & French, 2019). Beyond statistical significance, however, only a very small proportion of the total variability in student standardized achievement was related to ratings of teacher instructional practices—at most, 2.5% for reading achievement and 1.3% for mathematics achievement.…”

Section: Discussionsupporting

confidence: 87%

The Predictive Validity of Classroom Observations: Do Teachers’ Framework for Teaching Scores Predict Kindergarteners’ Achievement and Motivation?

Patrick¹,

Mantzicopoulos

French

2019

American Educational Research Journal

Self Cite

View full text Add to dashboard Cite

We used multilevel analysis to examine the predictive validity of scores from the Framework for Teaching (FFT), the observation measure used most often to evaluate teachers’ instruction. We investigated how well 81 kindergarten teachers’ FFT scores for eight reading and eight mathematics lessons observed throughout the year predicted students’ year-end achievement and motivation in reading and mathematics, controlling for students’ sex, ethnicity, and achievement entering kindergarten. Standardized reading and mathematics achievement were each predicted by FFT scores; however, they accounted for very little of the overall variance in students’ achievement: 2.5% for reading and 1.3% for mathematics. Neither students’ end-of-year criterion-referenced achievement nor motivation were predicted by FFT scores.

show abstract

Section: Discussionsupporting

confidence: 87%

The Predictive Validity of Classroom Observations: Do Teachers’ Framework for Teaching Scores Predict Kindergarteners’ Achievement and Motivation?

Patrick¹,

Mantzicopoulos

French

2019

American Educational Research Journal

Self Cite

View full text Add to dashboard Cite

show abstract

“…The purpose, methodology by way of GT application and the first finding of this study are in line with those of a study by Patrick et al (2020), which evaluated the score stability of the Framework for Teaching (FFT). G coefficients for relative interpretation (Eρ 2 ) had a range of 0.92 to 0.96 for the FFT's classroom environment and total scores, and were 0.87 and 0.79 for reading and mathematics instruction correspondingly.…”

Section: Data Processing and Analysissupporting

confidence: 59%

“…Many empirical studies have been conducted worldwide on instrument development with the use of GT focusing on reliability investigation. Notable among them is the study by Patrick et al (2020) with the aim of assessing the score stability of the Framework for Teaching (FFT). The FFT is an observation instrument used for teacher assessment in the USA.…”

Section: Empirical Reviewmentioning

confidence: 99%

“…A lot of concerns have been raised about the validity and reliability of classroom observational data (Weisberg et al, 2009). In the USA, despite the proliferation of the use of these observation instruments in the school system, there is insufficient evidence of the reliability of their scores and other psychometric properties of such instruments (Patrick et al, 2020). In a similar vein, a query of the archives of some teacher training institutions in Ghana failed to find any evidence of the psychometric properties of their teaching practice rating scales (Oduro-Okyireh, 2020).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Reliability of Students’ Teaching Practice Scores Using Generalizability Theory

Oduro-Okyireh,

Asamoah-Gyimah,

Oduro-Okyireh

et al. 2024

AJAR

View full text Add to dashboard Cite

Purpose: The aim was to demonstrate the potency of using Generalisability Theory (GT) over Classical Test Theory (CTT) in reliability estimation. The objectives are to find the major source of error in the Intern Teaching Evaluation Form (ITEF) scores and to identify the optimum number of occasions of rating of teaching practice that would give the most reliable scores. Design/ Methodology/ Approach: A random effects one-facet fully crossed design in which intern (p) was fully crossed with occasion (o) was adopted for the study. In all, 9,082 bachelor’s degree teaching practice triplicate scores for three academic years from 2015/2016 to 2017/2018 were analysed in this study. A univariate generalisability analysis with EduG was performed to analyse data. Findings: The finding for relative interpretation, the scores were strongly reliable. For absolute interpretation, the scores were moderate to strongly dependable. The major source of error in the ITEF scores was the p x o interaction combined with unidentified sources. Research Limitation/Implications: A firm conclusion on one major source of error being a single facet in the ITEF scores could not be made. Practical Implication: The level of reliability and the optimum scoring design of the ITEF established by this study will aid in selecting the ITEF by teacher training institutions in Ghana for evaluation of teaching practice to train pre-service teachers for effective curriculum implementation to achieve national educational aims. Social Implication: Reliable evaluation data can inform policy decisions related to teacher certification, professional development, and educational standards. Policymakers can use this data to develop evidence-based policies that support teacher quality and student achievement. Originality/ Value: It has unearthed the psychometric properties and the ideal scoring design of the ITEF which were hitherto unknown.

show abstract

“…A lot of uncertainties have been raised concerning the structural validity and reliability of classroom observational data (Galvão et al, 2021;Hill et al, 2012). Patrick et al (2020) assert that, in the USA, in spite of the explosion of the use of observation instruments in the school system nationwide, there is insufficient evidence of the validity and reliability of the instruments, including other psychometric properties of such scales which are so important to aid decision making. Oduro-Okyireh (2020) emphasises that, this is apparently not different from the Ghanaian situation, since the field of psychometry is relatively an emerging area in psychology and education in Ghana.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Teaching Practice: Assessment of the Dimensionality of the Intern Teaching Evaluation Form (ITEF)

Oduro-Okyireh

2024

BJE

View full text Add to dashboard Cite

This study aimed at investigating the dimensionality of the Intern Teaching Evaluation Form (ITEF), which is an observation instrument used in the evaluation of teaching practice at a public university in Ghana. The instrument was used to assess the teaching skills of a group of interns across three lessons. Data were obtained over a three-year period from 2016/2017 to 2018/2019 academic years during the Students Internship Programme (SIP) of the university. Part of the overall aim was to find the score stability (reliability) of the ITEF when it is used to evaluate teaching skill. A random effect two facets fully crossed design where intern (t) was crossed with item (i) and lesson (l), given by (t×i×l) was adopted for the dimensionality investigation. Further, a random effect one-facet fully crossed design, where inter (t) was crossed with lesson (l), given by (t×l) was adopted for the reliability investigation. The sample sizes for the study were 9132 undergraduate bachelor’s degree ITEF triplicate scores for the reliability investigation and 50 ITEF triplicate scores for the dimensionality investigation, selected by purposive and simple random sampling methods respectively. Univariate generalisability analysis using EduG was performed to analyse data. Findings were that, the ITEF is not unidimensional, but highly reliable (strongly stable) across the delivery of three lessons. Caution should be exercised in taking the ITEF scores as unidimensional in running correlational analyses with other criteria, unless this is done only from the perspective of ‘use,’ which should be one of the important considerations in instrument design.

show abstract

The Reliability of Framework for Teaching Scores in Kindergarten

Cited by 8 publications

References 15 publications

The Predictive Validity of Classroom Observations: Do Teachers’ Framework for Teaching Scores Predict Kindergarteners’ Achievement and Motivation?

The Predictive Validity of Classroom Observations: Do Teachers’ Framework for Teaching Scores Predict Kindergarteners’ Achievement and Motivation?

Reliability of Students’ Teaching Practice Scores Using Generalizability Theory

Evaluation of Teaching Practice: Assessment of the Dimensionality of the Intern Teaching Evaluation Form (ITEF)

Contact Info

Product

Resources

About