Argus: Interactive a priori Power Analysis

Wang, Xiaoyi; Eiselmayer, Alexander; Mackay, Wendy E.; Hornbæk, Kasper; Wacharamanotham, Chat

doi:10.1109/tvcg.2020.3028894

Cited by 6 publications

(4 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The practice is more solid for quantitative research (justify-n-qan) as researchers can conduct power analysis and calculate the minimum required sample size to acquire significant findings with a specific level of power and acceptable effect size [26,34]. Support for these processes can come from software such as G*Power and other innovations for sample size computation [46]. For qualitative research (justify-n-qal), the most common justification for sample size is saturation [8] where researchers recruit participants until they reach saturation in their qualitative analysis (i.e., participants are no longer revealing new discussion topics).…”

Section: Practices Related To Participantsmentioning

confidence: 99%

Criteria Definition for Research Ethics, Openness, and Transparency

Niksirat¹,

Goswami²,

Rao³

et al. 2023

Preprint

View full text Add to dashboard Cite

This document is supplementary material for the paper “Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022”; published at ACM CHI 2023. The document provides the rationale behind each criterion in detail with additional citations. We hope that knowing the rationale will better encourage practices related to research ethics,openness, and transparency.The original paper and paper and all supplementary materials are freely available at: https://doi.org/10.17605/osf.io/n25d6

show abstract

Section: Practices Related To Participantsmentioning

confidence: 99%

Criteria Definition for Research Ethics, Openness, and Transparency

Niksirat¹,

Goswami²,

Rao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…There will be hands-on practice in identifying ambiguities and omissions of research decisions from excerpts of research papers. We will discuss strategies for planning sample sizes [23], and the participants will explore how their decisions impact the statistical power using a simulation tool (e.g., [51]).…”

Section: Planning Research and Sharing Research Artifactsmentioning

confidence: 99%

“…Chat Wacharamanotham is an Assistant Professor at the University of Zurich (UZH). The focus of his work is on understanding and developing tools for planning, reporting, reading, and sharing quantitative research [12,32,51,59,60]. He is also a co-organizer of the Transparent Statistics in Human-Computer Interaction group.…”

Section: Instructorsmentioning

confidence: 99%

Transparent Practices for Quantitative Empirical Research

Wacharamanotham

Yang

et al. 2022

CHI Conference on Human Factors in Computing Systems Extended Abstracts

Self Cite

View full text Add to dashboard Cite

Transparent research practices analytic methods, and data to be thoroughly evaluated and potentially reproduced. The HCI community has recognized research transparency as one quality aspect of paper submission and review since CHI 2021. This course addresses HCI researchers and students who are already knowledgeable about experiment research design and statistical analysis. Building upon this knowledge, we will present current best practices and tools for increasing research transparency. We will cover relevant concepts and skills in Open Science, frequentist statistics, and Bayesian statistics, and uncertainty visualization. In addition to lectures, there will be hands-on exercises: The course participants will assess transparency practices in excerpts of quantitative reports, interactively explore implications of analytical choices using RStudio Cloud, and discuss their fndings in small groups. In the fnal session, each participant will choose a case study based on their interest and assess its research transparency together with their classmates and instructors.enable the research design, materials, CCS CONCEPTS• Human-centered computing → HCI design and evaluation methods; Visualization techniques; Visualization design and evaluation methods.

show abstract

“…For example, there is the R package pwr [2], software G*Power [10], and interactive web-based tool Touchstone2 [9]. The knowledge of the small, medium and large eect sizes allows the researchers to enter all of them using these tools and assess the design trade-os [42]. Using all three reference eect sizes and then assessing whether the sample size is reasonable can be a way to do a priori power analysis.…”

Section: How To Use the Ruler?mentioning

confidence: 99%

A Meta-Analysis of Effect Sizes of CHI Typing Experiments

Obukhova

2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

While designing an HCI experiment, planning the sample size with a priori power analysis is often skipped due to the lack of reference effect sizes. On the one hand, it can lead to a false-negative result, missing the effect that is present in the population. On the other hand, it poses a risk of spending more resources if the number of participants is too high. In this work, I present the reference for small, medium, and large effect sizes for typing experiments based on a meta-analysis of well-cited papers from CHI conference. This effect size ruler can be used to conduct a priori power analysis or assess the magnitude of the found effect. This work also includes comparisons to other fields and conclude with a discussion of the existing issues with reporting practices and data availability. This paper and all data and materials are freely available at https://osf.io/nqzpr.

show abstract

Argus: Interactive a priori Power Analysis

Cited by 6 publications

References 52 publications

Criteria Definition for Research Ethics, Openness, and Transparency

Criteria Definition for Research Ethics, Openness, and Transparency

Transparent Practices for Quantitative Empirical Research

A Meta-Analysis of Effect Sizes of CHI Typing Experiments

Contact Info

Product

Resources

About