Replication and Evidence Factors in Observational Studies

Rosenbaum, Paul R.

doi:10.1201/9781003039648

Cited by 18 publications

(10 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since causal inference in observational data relies on the plausibility of the assumptions required for each method used, multiple methods can be used with different assumptions and results compared-termed 'triangulation' 93 in epidemiology or comparison of 'evidence factors' in social science. 94 Even where the same magnitude of association is found in different studies, they may still reflect differences in the underlying processes in each study; for instance, a causal effect in one study, yet a confounded effect in another. Thus, care is also required to account for such causes of bias even where no cross-study differences are reported.…”

Section: Sources Of Different Results Between Cohortsmentioning

confidence: 99%

Investigating change across time in prevalence or association: the challenges of cross-study comparative research and possible solutions

Bann¹,

Wright²,

Goisis³

et al. 2021

Preprint

View full text Add to dashboard Cite

Across the health and social sciences, addressing many key scientific or policy questions requires an understanding of whether a given quantity has changed over time—e.g., by year of data collection or by birth year. For example, has the occurrence of—or socioeconomic inequity in—a given health outcome changed across time? Or has social mobility improved or worsened in recently born generations? Understanding such issues motivates and informs future policy development, and can provide clues to aetiology by contrasting with changes in purported determinants or confounding factors. Accordingly, comparative or cross-cohort research initiatives are increasingly prominent components of health and social sciences yet, to our knowledge, no structured guidance exists to inform the implementation or appraisal of such studies. We thus collate relevant learning and produce guidance, focusing on changes across time in 1) outcome occurrence or levels for continuous outcomes; and 2) the magnitude and/or direction of associations. We discuss their use and importance (when does such comparative research add value?), study inclusion (which studies could be used?), sources of bias (what can go wrong and how can biases be mitigated?), and interpretation (how can between study differences be explained?). We provide a checklist to structure such guidance which applies to multiple study designs (e.g., cohort or repeated cross-sectional studies) and may be a useful tool for future authors and reviewers focusing on comparative research across time, cohort or indeed place. Finally, we provide analytical syntax and tutorial content with example data to facilitate future analysis and data visualisation (see https://osf.io/d569x/).

show abstract

Section: Sources Of Different Results Between Cohortsmentioning

confidence: 99%

Investigating change across time in prevalence or association: the challenges of cross-study comparative research and possible solutions

Bann¹,

Wright²,

Goisis³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

“…Finally, this technique can also be used for the construction of matched samples with evidence factors, where each factor requires a separate balanced matched contrast for a given covariate profile. 61 Any statistical method for adjustment has advantages and disadvantages. While profile matching, in preserving the unit of analysis, can facilitate a simple interpretation of the adjusted sample, weighting methods can often achieve greater statistical efficiency and be more computationally tractable.…”

Section: Discussionmentioning

confidence: 99%

Profile Matching for the Generalization and Personalization of Causal Inferences

Cohn

Zubizarreta

2022

Epidemiology

View full text Add to dashboard Cite

We introduce profile matching, a multivariate matching method for randomized experiments and observational studies that finds the largest possible unweighted samples across multiple treatment groups that are balanced relative to a covariate profile. This covariate profile can represent a specific population or a target individual, facilitating the generalization and personalization of causal inferences. For generalization, because the profile often amounts to summary statistics for a target population, profile matching does not always require accessing individual-level data, which may be unavailable for confidentiality reasons. For personalization, the profile comprises the characteristics of a single individual. Profile matching achieves covariate balance by construction, but unlike existing approaches to matching, it does not require specifying a matching ratio, as this is implicitly optimized for the data. The method can also be used for the selection of units for study follow-up, and it readily applies to multivalued treatments with many treatment categories. We evaluate the performance of profile matching in a simulation study of the generalization of a randomized trial to a target population. We further illustrate this method in an exploratory observational study of the relationship between opioid use and mental health outcomes. We analyze these relationships for three covariate profiles representing: (i) sexual minorities, (ii) the Appalachian United States, and (iii) the characteristics of a hypothetical vulnerable patient. The method can be implemented via the new function profmatch in the designmatch package for R, for which we provide a step-by-step tutorial.

show abstract

“…For related discussion, see Rosenbaum (2001Rosenbaum ( , 2021, Lawlor et al (2016) and Munafò and Davey-Smith (2018).…”

Section: Strengthening Causal Inference By Integrating Methodologies 1effects Caused By Treatmentsmentioning

confidence: 99%

Optimal Matching for Observational Studies That Integrate Quantitative and Qualitative Research

Small

Harding

et al. 2021

Statistics and Public Policy

Self Cite

View full text Add to dashboard Cite

A quantitative study of treatment effects may form many matched pairs of a treated subject and an untreated control who look similar in terms of covariates measured prior to treatment. When treatments are not randomly assigned, one inevitable concern is that individuals who look similar in measured covariates may be dissimilar in unmeasured covariates. Another concern is that quantitative measures may be misinterpreted by investigators in the absence of context that is not recorded in quantitative data. When text information is automatically coded to form quantitative measures, examination of the narrative context can reveal the limitations of initial coding efforts. An existing proposal entails a narrative description of a subset of matched pairs, hoping in a subset of pairs to observe quite a bit more of what was not quantitatively measured or automatically encoded. A subset of pairs cannot rule out subtle biases that materially affect analyses of many pairs, but perhaps a subset of pairs can inform discussion of such biases, perhaps leading to a reinterpretation of quantitative data, or perhaps raising new considerations and perspectives. The large literature on qualitative research contends that open-ended, narrative descriptions of a subset of people A c c e p t e d M a n u s c r i p tcan be informative. Here, we discuss and apply a form of optimal matching that supports such an integrated, quantitative-plus-qualitative study. The optimal match provides many closely matched pairs plus a subset of exceptionally close pairs suitable for narrative interpretation. We illustrate the matching technique using data from a recent study of police responses to domestic violence in Philadelphia, where the police report includes both quantitative and narrative information.

show abstract

Replication and Evidence Factors in Observational Studies

Cited by 18 publications

References 16 publications

Investigating change across time in prevalence or association: the challenges of cross-study comparative research and possible solutions

Investigating change across time in prevalence or association: the challenges of cross-study comparative research and possible solutions

Profile Matching for the Generalization and Personalization of Causal Inferences

Optimal Matching for Observational Studies That Integrate Quantitative and Qualitative Research

Contact Info

Product

Resources

About