A Framework for Addressing Instrumentation Biases When Using Observation Systems as Outcome Measures in Instructional Interventions

White, Mark; Maher, Bridget; Rowan, Brian

doi:10.1080/19345747.2022.2081275

Cited by 5 publications

(5 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, it is also the first to use video data to systematically track secondary‐school students' technology use in real classroom settings over time, contributing unique baseline data to future research to determine a baseline regarding digital skills and practices based on a large student sample (cf. Creswell & Creswell, 2018; Tashakkori et al, 2020; White et al, 2022a). Video data provide opportunities for researchers to observe raw data multiple times and discern details that other data types do not capture (Klette, 2020).…”

Section: Methodsmentioning

confidence: 99%

Section: Need For Systematic Video Studies Of Technology Uptakementioning

confidence: 99%

“…While this can be valuable in small case studies, it is also possible to gather and compare data across lessons, subjects and even countries. In the United States, the Measures of Effective Teaching (MET) project (MET Project, 2009; see also Kane & Staiger, 2012) collected 20,000 videotaped lessons over a period of 3 years (2010)(2011)(2012)(2013), to study the construct of teaching quality across subjects and classrooms (e.g., Blazar & Kraft, 2017;Briggs & Alzen, 2019;Campbell & Ronfeldt, 2018;Cantrell & Kane, 2013;Jensen et al, 2018;Kane & Cantrell, 2010;Schultz & Pecheone, 2014;White et al, 2022aWhite et al, , 2022b.…”

Section: Need For Systematic Video Studies Of Technology Uptakementioning

confidence: 99%

“…Thus, this work aims to contribute unique data to future research to determine a baseline regarding digital skills and practices based on a large student sample (cf. Creswell & Creswell, 2018; Tashakkori et al, 2020; White et al, 2022a, 2022b). The aim is to give perspective on classroom technology use and to broaden the research interest in video observations, revealing how digital skills are used in authentic English classroom settings.…”

Section: Need For Systematic Video Studies Of Technology Uptakementioning

confidence: 99%

See 3 more Smart Citations

Digital skills critical for education: Video analysis of students' technology use in Norwegian secondary English classrooms

Kure

Brevik

Blikstad‐Balas

2022

Computer Assisted Learning

View full text Add to dashboard Cite

Background: Globally, digital skills are a crucial aspect of education that schools should develop systematically. Research on digital skills tends to be measured using self-reports, performance tests or interventions. There is less knowledge about student and teacher uptake of technology in school, making it important to investigate the actual use of technology and digital skills in authentic classroom settings.Objectives: This study contributes unique baseline data concerning students' use of technology and digital skills across mandatory English courses in real classroom settings in secondary schools in Norway over time. Methods:The study adapted a national framework for digital skills into an observation protocol. With it, this study analysed videos from 60 naturally occurring English lessons in 13 English classes at seven lower secondary schools over two school years (grades 9 and 10), following 186 students (aged 13-15) and 10 teachers. Results and Conclusions: Students used digital skills critical for education in half of the video-recorded English lessons, with more digital skill use at some schools over time. The main finding across classrooms and school years regards students' use of basic, not advanced, digital skills. Takeaways: Although teachers provide opportunities for students to use digital skills in school, more advanced skills are needed. This work calls for continued use of video recordings to provide systematic comparisons of potential shifts in students' digital skills in real English secondary classroom settings over time.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Need For Systematic Video Studies Of Technology Uptakementioning

confidence: 99%

Section: Need For Systematic Video Studies Of Technology Uptakementioning

confidence: 99%

Section: Need For Systematic Video Studies Of Technology Uptakementioning

confidence: 99%

See 2 more Smart Citations

Digital skills critical for education: Video analysis of students' technology use in Norwegian secondary English classrooms

Kure

Brevik

Blikstad‐Balas

2022

Computer Assisted Learning

View full text Add to dashboard Cite

show abstract

“…First, in-person observations are susceptible to observer bias (Hunter, 2020 ; White et al, 2022 ), which refers to systematic deviations from the truth that occur due to observer and contextual characteristics (Mahtani et al, 2018 ). For instance, evidence suggests that an observer's emotional state can bias observation ratings (Floman et al, 2016 ).…”

Section: Accuracy Of Automated Processing Of Child and Adult Language...mentioning

confidence: 99%

Evaluating the accuracy of automated processing of child and adult language production in preschool classrooms

Pelfrey,

Justice,

Gonzalez Villasanti

et al. 2024

Front. Psychol.

View full text Add to dashboard Cite

Young children's language and social development is influenced by the linguistic environment of their classrooms, including their interactions with teachers and peers. Measurement of the classroom linguistic environment typically relies on observational methods, often providing limited 'snapshots' of children's interactions, from which broad generalizations are made. Recent technological advances, including artificial intelligence, provide opportunities to capture children's interactions using continuous recordings representing much longer durations of time. The goal of the present study was to evaluate the accuracy of the Interaction Detection in Early Childhood Settings (IDEAS) system on 13 automated indices of language output using recordings collected from 19 children and three teachers over two weeks in an urban preschool classroom. The accuracy of language outputs processed via IDEAS were compared to ground truth via linear correlations and median absolute relative error. Findings indicate high correlations between IDEAS and ground truth data on measures of teacher and child speech, and relatively low error rates on the majority of IDEAS language output measures. Study findings indicate that IDEAS may provide a useful measurement tool for advancing knowledge about children's classroom experiences and their role in shaping development.

show abstract

Drawing on Siedentop’s Legacy to Revisit Systematic Observation of Teaching: Identifying and Addressing Key Validity and Reliability Questions

Tsangaridou

Charalambous

2023

Quest

View full text Add to dashboard Cite

A Framework for Addressing Instrumentation Biases When Using Observation Systems as Outcome Measures in Instructional Interventions

Cited by 5 publications

References 52 publications

Digital skills critical for education: Video analysis of students' technology use in Norwegian secondary English classrooms

Digital skills critical for education: Video analysis of students' technology use in Norwegian secondary English classrooms

Evaluating the accuracy of automated processing of child and adult language production in preschool classrooms

Drawing on Siedentop’s Legacy to Revisit Systematic Observation of Teaching: Identifying and Addressing Key Validity and Reliability Questions

Contact Info

Product

Resources

About