Scientific Utopia III: Crowdsourcing Science

Uhlmann, Eric Luis; Ebersole, Charles R.; Chartier, Christopher R.; Errington, Tim; Kidwell, Mallory C.; Lai, Calvin K.; McCarthy, Randy J.; Riegelman, Amy; Silberzahn, Raphael; Nosek, Brian A.

doi:10.1177/1745691619850561

Cited by 129 publications

(113 citation statements)

References 161 publications

(205 reference statements)

Supporting

Mentioning

106

Contrasting

Unclassified

Order By: Relevance

“…Critically, authors can continuously revise their manuscripts based on the comments received, and the platform will maintain all versions of a manuscript, allowing scientists (and the public) to see how manuscripts evolve over months or even decades. Finally, nimbler publishing platforms designed for the digital age could crowdsource even peer-reviewing (for crowdsourcing research, see Landy et al, 2020;Uhlmann et al, 2019), allowing for continuous, rich, and scholarly discussions while ensuring transparency and rigor (e.g., Stern & O'Shea, 2019).…”

Section: Promises and Perils Of Experimentation 16mentioning

confidence: 99%

Promises and perils of experimentation: The mutual internal validity problem

Lin¹,

Werner²,

Inzlicht³

2020

Preprint

View full text Add to dashboard Cite

Experimentation is the cornerstone of the scientific method. Decades of experimental work in psychology, however, has yet to produce reliable and cumulative knowledge. We argue psychology as a discipline not only overuses experimentation, but too often uses it as a theatrical storytelling device. Rather than use experiments to facilitate valid scientific inference, psychologists cleverly design experiments to elicit the very effects they wish to observe, ignoring how these effects may or may not cohere with bodies of knowledge within psychology and across disciplines. This approach leads to ungeneralizable results and an unwillingness to accept and learn from failed, but potentially informative experiments. To make real progress toward understanding human behavior and the mind, psychology needs to embrace big-interdisciplinary (big-I) triangulation, which requires integrating theories, methods, and approaches from diverse disciplines and sources.

show abstract

Section: Promises and Perils Of Experimentation 16mentioning

confidence: 99%

Promises and perils of experimentation: The mutual internal validity problem

Lin¹,

Werner²,

Inzlicht³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In contrast to scientists, they will surely demand for specific answers which can have a clear value on their daily life. These practices embrace the perspective of a horizontally distributed scientific research, envisioned by Uhlmann et al (Uhlmann et al, 2019) as a way to accelerate the progress of science. Furthermore, as discussed above, standard experimental spaces in Computational Social Science are secluded or online laboratories that aim to neutralize the influence of the environment.…”

Section: Social Impactmentioning

confidence: 99%

Reformulating computational social science with citizen social science: the case of a community-based mental health care research

Bonhoure¹,

Cigarini²,

Vicens³

et al. 2019

Preprint

View full text Add to dashboard Cite

“Citizen Social Science” is a fast-growing term within Citizen Science world but not many experiences have been shared in the literature to better shape its meaning. This article discusses novel possibilities of these participatory practices through the critical analysis of a concrete community-based project. We here embrace under the “Social” tag of Citizen Social Science both Computational Social Science methodologies and the Social concerns expressed by the mental health community. The interpretation place persons with a mental condition, as well as caregivers and relatives, at the center of the research cycle by taking the role of Competent Experts In-The-Field. A synergetic relation between Citizen Science, Mental Health research and Computational Social Science also imply a conceptual shift in comparison to standard approaches such as the inclusion of research subjects as active co-designers or the consideration of communitarian spaces as most natural experimental spaces. We here describe how these concepts are put into practice during the whole duration of a research project that has studied the social interactions inside the Community Mental Health Care ecosystem. Important steps entail the creation of a Knowledge Coalition, including a diversity of relevant actors with diverse knowledge and expertise, that is later involved in co-designing the research and in embedding experimental settings in communitarian spaces. The experience allows us to open a wider discussion on the possibilities and limitations of Citizen Social Science practices. Having in mind the ethical debate raised by Citizen Science Public Health and Patients’ driven research, we propose a set of values and practices to be agreed on. We also analyse the participation of the scientists in these kind of projects, which forces them to adopt the role of “camaleons” when executing diverse and versatile tasks. We additionally advocate for a more extended collective interpretation of the results in order to produce socially robust knowledge and enhance actions and policies grounded on these results. Exemplified with the experience herein presented, a more horizontal process that include the enhancement of participation and the revision of the notion of experiment offer new opportunities for Social Sciences from a multidisciplinary perspective.

show abstract

“…Broadly, persistent collaboration consortia could help more efficiently allocate existing resources, reduce barriers to entry, and increase inclusivity, while also increasing transparency, rigor, and reliability of the results produced (Uhlmann et al, 2019). Establishing consortia provides many advantages, including larger sample sizes, more diversity among potential participants, increased depth and breadth of available expertise, and greater inclusivity within the research process.…”

Section: Persistent Collaborationmentioning

confidence: 99%

A Path to Greater Credibility: Large-Scale Collaborative Education Research

et al. 2019

View full text Add to dashboard Cite

Concerns about the replication crisis and unreliable findings have spread through several fields, including education and psychological research. In some areas of education, researchers have begun to adopt reforms that have proven useful in other fields. These include preregistration, open materials and data, and registered reports. These reforms offer education research a path toward increased credibility and social impact. In this article, we discuss models of large-scale collaborative research practices and how they can be applied to education research. We discuss five types of large-scale collaboration: participating teams run different studies, multiteam collaboration projects, collaborative analysis, preregistered adversarial collaboration, and persistent collaboration. The combination of large-scale collaboration with open and transparent research practices offers education researchers opportunity to test theories, verify what is known about a topic, resolve disagreements, and explore new questions.

show abstract

Scientific Utopia III: Crowdsourcing Science

Cited by 129 publications

References 161 publications

Promises and perils of experimentation: The mutual internal validity problem

Promises and perils of experimentation: The mutual internal validity problem

Reformulating computational social science with citizen social science: the case of a community-based mental health care research

A Path to Greater Credibility: Large-Scale Collaborative Education Research

Contact Info

Product

Resources

About