Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

Abstract

Discussions around transparency in open science focus primarily on sharing data, materials, and coding schemes, especially as these practices relate to reproducibility. This fairly quantitative perspective of transparency does not align with all scientific methodologies. Indeed, qualitative researchers also care deeply about how knowledge is produced, what factors influence the research process, and how to share this information. Explicating a researcher’s background and role allows researchers to consider their impact on the research process and interpretation of the data, thereby increasing both transparency and rigor. Researchers may engage in positionality and reflexivity in a variety of ways, and transparently sharing these steps allows readers to draw their own informed conclusions about the results and study as a whole. Imposing a limited, quantitatively-informed set of standards on all research can cause harm to researchers and the communities they work with if researchers are not careful in considering the impact of such standards. Our paper will argue the importance of avoiding strong defaults around transparency (e.g., always share data) and build upon previous work around qualitative open science. We explore how transparency in all aspects of our research can lend itself toward projecting and confirming the rigor of our work.

Open-Science Guidance for Qualitative Research: An Empirically Validated Approach for De-Identifying Sensitive Narrative Data

“…Such framing has drawn criticism from qualitative scholars because narrative inquiry is grounded in constructivist or critical epistemologies, without ontological assumptions of an objective reality (Lincoln et al, 2018). Qualitative research seeks to understand how people interpret their lived experiences, recognizing that such knowledge is bounded by history and context (Field et al, 2021;Steltenpohl et al, 2023). There is no expectation that a different analyst would necessarily interpret data the same way or that another researcher pursuing the same or similar research questions in another setting would obtain the same results (Tsai et al, 2016).…”

“…Given these essential differences, many qualitative scholars remain skeptical about whether reproducibility, replication, and/or reuse are sensible goals for this type of inquiry (Bennett, 2021;Brabeck, 2021;Feldman & Shaw, 2019;Parry & Mauthner, 2004). However, other qualitative researchers maintain that the open-science movement's guiding principles can be reimagined and tailored to narrative inquiry (Class et al, 2021;DuBois et al, 2018;Steltenpohl et al, 2023). As these debates have been unfolding in the literature, the expectations for greater transparency have grown stronger.…”

“…We begin by reviewing key epistemological, ethical, and methodological concerns that researchers must address if they intend to share qualitative data. It is beyond the scope of this article to unpack each of these sizable dilemmas in depth (for reviews, see DuBois et al, 2018;Field et al, 2021;Steltenpohl et al, 2023), so in our review, we highlight the tensions between promoting transparency and protecting privacy.…”

Qualitative research at the crossroads of open science and big data: Ethical considerations

“…However, as Ricker (2017) notes, big data "may seem overtly quantitative, [but] the data are also overwhelmingly qualitative in nature, necessitating methodology distinctive to qualitative research" (p. 2). This then raises questions about how these guidelines and recommendations might be appropriately applied to qualitative, mixed methods, and other research traditions (Steltenpohl et al, 2023;Weigelt et al, 2022). These reforms may seem like positive solutions to counteract reports of questionable research practices that have been noted in both qualitative and quantitative research, such as selective reporting or extrapolating results beyond evidence/findings (e.g., Antonio et al, 2020;Cook et al, 2018;Fraser et al, 2018;John et al, 2012;Makel et al, 2021;Perazzo et al, 2019).…”

“…However, most discussions focus on ethics relevant to quantitative, lab-based experimental approaches, and computational code sharing. Some discussions of ethics related to transparency, openness, data sharing, and reproducibility beyond quantitative approaches have emerged and initiated critical conversations about open science for other research traditions (e.g., Ananny & Crawford, 2018;Class et al, 2021;Hocker et al, 2021;Makel et al, 2022;Steltenpohl et al, 2021;Steltenpohl et al, 2023). Yet, further detail on how to ethically and effectively integrate these changes into current policies, systems, and individual practices is needed.…”