A note on replication analysis

Hoeppner, Sven

doi:10.1016/j.irle.2019.05.004

Cited by 5 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are some existing statistical tools for conducting a replication analysis. For example, if researchers are curious about the significance of effect sizes, they may conduct two hypothesis testings, one for the original study and the other for replication study, which might be called as "vote counting" (Hoeppner, 2019). However, as pointed out (Valentine et al, 2011), there can be issues in the interpretation on the probabilistic sides, since the p-value in one study doesn't have the same meaning for the other one.…”

Section: Review Of Replication Analysismentioning

confidence: 99%

“…However, as pointed out (Valentine et al, 2011), there can be issues in the interpretation on the probabilistic sides, since the p-value in one study doesn't have the same meaning for the other one. Misleading results can even happen under underpowered replication studies (Simonsohn, 2015;Hoeppner, 2019). If researcher are curious about the magnitude of effect sizes, they may compare the two point estimations for the effect of interest, while there is an increasing tendency for using confidence intervals rather than point estimations.…”

Section: Review Of Replication Analysismentioning

confidence: 99%

“…However, the methods merely using frequentist point estimations or hypothesis testing are under criticism recently, and more emphasis is being put on methods incorporating uncertainty quantifications. In addition, Bayesian approach is getting more attention (Hoeppner, 2019): because of the technical restriction of frequentist approaches and the epistemic goal of replication analysis.…”

Section: Review Of Replication Analysismentioning

confidence: 99%

“…Since we're no longer comparing a new estimator to a constant but comparing two parameters through comparing two estimators under repeated experiments, it's intuitive to compare their confidence intervals (CI), which can also be preferred by replication analysis community (Hoeppner, 2019). There are many advantages by using CI, one of which is that it gives better probabilistic interpretation than using an arbitrarily set tolerance region, since CI provides the probability of covering the true parameter under repeated experiments.…”

Section: The Framework For Alternative Model (Am)mentioning

confidence: 99%

“…Apart from the direct desire to repeat and check a published conclusion, replication analysis can also be motivated by different reasons. A possible reason is some changes on the side of data (Hoeppner, 2019). For some time-sensitive conclusions, researchers may want to conduct a replication study on current data to see whether the results in the research published long time ago still hold or not.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Differentially Private Bayesian Approach to Replication Analysis

Yang¹,

Reiter²

2021

Preprint

View full text Add to dashboard Cite

Replication analysis is widely used in many fields of study. Once a research is published, many other researchers will conduct the same or very similar analysis to confirm the reliability of the published research. However, what if the data is confidential? In particular, if the data sets used for the studies are confidential, we cannot release the results of replication analyses to any entity without the permission to access the data sets, otherwise it may result in serious privacy leakage especially when the published study and replication studies are using similar or common data sets. For example, examining the influence of the treatment on outliers can cause serious leakage of the information about outliers. In this paper, we build two frameworks for replication analysis by a differentially private Bayesian approach. We formalize our questions of interest and illustrates the properties of our methods by a combination of theoretical analysis and simulation to show the feasibility of our approach. We also provide some guidance on the choice of parameters and interpretation of the results.

show abstract

Section: Review Of Replication Analysismentioning

confidence: 99%

Section: Review Of Replication Analysismentioning

confidence: 99%

Section: Review Of Replication Analysismentioning

confidence: 99%

Section: The Framework For Alternative Model (Am)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Differentially Private Bayesian Approach to Replication Analysis

Yang¹,

Reiter²

2021

Preprint

View full text Add to dashboard Cite

show abstract

Relating utterance fluency to perceived fluency of interpreting

Han

Yang

2022

TIS

View full text Add to dashboard Cite

The prospect of automated scoring for interpreting fluency has prompted investigations into the predictability of human raters’ perceived fluency based on acoustically measured utterance fluency. Recently, Han, Chen, Fu and Fan (2020) correlated ten utterance fluency measures with raters’ perceived fluency ratings. To verify previous correlational patterns, the present study partially replicated Han et al. (2020). Our analysis shows that most of the correlations observed in Han et al. (2020) were successfully replicated. To produce overall interim estimates of the true relationships, we conducted a mini meta-analysis of correlation coefficients reported in six relevant studies, informed by the “continuously cumulating meta-analysis” approach (Braver et al. 2014). We found that phonation time ratio, mean length of run, and speech rate had relatively strong correlations with perceived fluency. We discuss these findings in light of automated fluency assessment and the need for replication and meta-analysis in translation and interpreting studies.

show abstract

Differentially Private Methods for Releasing Results of Stability Analyses

Yang,

Reiter

2023

The American Statistician

View full text Add to dashboard Cite

A note on replication analysis

Cited by 5 publications

References 21 publications

A Differentially Private Bayesian Approach to Replication Analysis

A Differentially Private Bayesian Approach to Replication Analysis

Relating utterance fluency to perceived fluency of interpreting

Differentially Private Methods for Releasing Results of Stability Analyses

Contact Info

Product

Resources

About