The Design of Clustered Observational Studies in Education

Page, Lindsay C.; Lenard, Matthew; Keele, Luke

doi:10.1177/2332858420954401

Cited by 13 publications

(13 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Our work also awaits replication. To corroborate and extend our findings and preliminary conclusions, further analysis could employ clustered observational studies and inferential statistics and also consider the specific development methods, application domains, team structures, and underlying organizational and individual cultural factors potentially affecting communication in software engineering teams [26], [93], [94]. This is a major issue in all scientific research and this is why we make our data openly available [95], [96].…”

Section: Limitations and Threats To Validitysupporting

confidence: 51%

Non Verbal Communication in Software Engineering – An Empirical Study

et al. 2021

View full text Add to dashboard Cite

Communication among humans consists of both verbal and non verbal components. The latter may sometimes express concepts or ideas not conveyable by the former. This is also true in Software Engineering. This paper first analyses theoretically the role of non verbal communication in software development teams, using the framework provided by distributed cognition as a conceptual palette and as a point of reference. Then, it presents an empirical investigation involving 38 IT professionals from Russia, sharing their experiences in communicating and interacting when developing software artifacts. The results of this empirical investigation are consistent with many of the ideas underlying a distributed approach to cognition. In addition, our findings provide valuable insights to make communication more effective in software development teams, while defining a new framework for follow-up studies.INDEX TERMS Software engineering, software development, non verbal communication, empirical studies.

show abstract

Section: Limitations and Threats To Validitysupporting

confidence: 51%

Non Verbal Communication in Software Engineering – An Empirical Study

et al. 2021

View full text Add to dashboard Cite

show abstract

“…With treatment assignment at level 2, potential confounder variables are located at the group level (e.g., school resources, student characteristics aggregated at the school level), and covariates at level 1 are not relevant. Methods for estimating treatment effects with cluster-level assignment are discussed in Hansen et al (2014), Keele et al (2020), and Page et al (2020).…”

mentioning

confidence: 99%

Causal Inference with Multilevel Data: A Comparison of Different Propensity Score Weighting Approaches

Fuentes

Lüdtke

Robitzsch³

2021

Multivariate Behavioral Research

View full text Add to dashboard Cite

Propensity score methods are a widely recommended approach to adjust for confounding and to recover treatment effects with non-experimental, single-level data. This article reviews propensity score weighting estimators for multilevel data in which individuals (level 1) are nested in clusters (level 2) and nonrandomly assigned to either a treatment or control condition at level 1. We address the choice of a weighting strategy (inverse probability weights, trimming, overlap weights, calibration weights) and discuss key issues related to the specification of the propensity score model (fixed-effects model, multilevel randomeffects model) in the context of multilevel data. In three simulation studies, we show that estimates based on calibration weights, which prioritize balancing the sample distribution of level-1 and (unmeasured) level-2 covariates, should be preferred under many scenarios (i.e., treatment effect heterogeneity, presence of strong level-2 confounding) and can accommodate covariate-by-cluster interactions. However, when level-1 covariate effects vary strongly across clusters (i.e., under random slopes), and this variation is present in both the treatment and outcome data-generating mechanisms, large cluster sizes are needed to obtain accurate estimates of the treatment effect. We also discuss the implementation of survey weights and present a real-data example that illustrates the different methods.

show abstract

“…Much research has focused on whether Catholic schools are more effective than public schools in fostering student achievement (Coleman et al, 1982;Hoffer et al, 1985;Coleman and Hoffer, 1987). Often a new reading program may be implemented in some schools but not in others (Page et al, 2020). Other interventions broadly seek to change entire school structures in hopes of improving chronically underperforming schools (Bryk et al, 2015;Mehta et al, 2012;McGuinn, 2006).…”

Section: Educationmentioning

confidence: 99%

“…In the first application, the empirical question of interest is whether a summer school reading intervention in Wake County, NC improved students' reading scores (Pimentel et al, 2018;Page et al, 2020). Specifically, in the summer 2013, the Wake County Public School System (WCPSS) selected myON, a computer-aided reading program, for use in the summer school program for elementary school students.…”

Section: Summer School Reading Interventionmentioning

confidence: 99%

See 1 more Smart Citation

Nonparametric identification of causal effects in clustered observational studies with differential selection

Ye¹,

Westling²,

Page³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The clustered observational study (COS) design is the observational study counterpart to the clustered randomized trial. In a COS, a treatment is assigned to intact groups, and all units within the group are exposed to the treatment. However, the treatment is non-randomly assigned. COSs are common in both education and health services research. In education, treatments may be given to all students within some schools but withheld from all students in other schools. In health studies, treatments may be applied to clusters such as hospitals or groups of patients treated by the same physician. In this manuscript, we study the identification of causal effects in clustered observational study designs. We focus on the prospect of differential selection of units to clusters, which occurs when the units' cluster selections depend on the clusters' treatment assignments. Extant work on COSs has made an implicit assumption that rules out the presence of differential selection. We derive the identification results for designs with differential selection and that contexts with differential cluster selection require different adjustment sets than standard designs. We outline estimators for designs with and without differential selection. Using a series of simulations, we outline the magnitude of the bias that can occur with differential selection. We then present two empirical applications focusing on the likelihood of differential selection.

show abstract

The Design of Clustered Observational Studies in Education

Cited by 13 publications

References 51 publications

Non Verbal Communication in Software Engineering – An Empirical Study

Non Verbal Communication in Software Engineering – An Empirical Study

Causal Inference with Multilevel Data: A Comparison of Different Propensity Score Weighting Approaches

Nonparametric identification of causal effects in clustered observational studies with differential selection

Contact Info

Product

Resources

About