Concerns have been growing about the veracity of psychological research. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions, or attempt to replicate prior research, in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time-limited), efficient (in terms of re-using structures and principles for different projects), decentralized, diverse (in terms of participants and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside of the network). The PSA and other approaches to crowdsourced psychological science will advance our understanding of mental processes and behaviors by enabling rigorous research and systematically examining its generalizability.

No abstract

No abstract

Interacting with the published literature ("knowledge consumption") and publishing new scientific findings ("knowledge production") are two key moments in the scientist's search for truth, and bias in either of these can distort what is known about an area of research. This dissertation details three studies conducted on researchers in psychology that together provide evidence of scientists' behaviors influencing these key moments of knowledge production and knowledge consumption. Methods:Psychologists were recruited to participate in each study (N = 215 and N = 587).Studies used custom web tools and social network methods to collect unique datasets on psychologists' social networks and how they approach the scientific literature. The analytic approach differed based on each study. For studies on knowledge consumption, Gini coefficients and measures of unpredictability were calculated to better understand the dynamics of the published literature. For studies on knowledge production, the generalized network scale up method was used to estimate the size of the population of iii current users of questionable research practices, and regression was used to better understand the relationship between attitudes and stigma against certain psychologists. Results:Chapter 1: Overview and IntroductionHistorians don't want to write a history of historians. They are quite happy to plunge endlessly into limitless historical detail. But they themselves don't want to be counted as part of the limitless historical detail. They don't want to be part of the historical order. It's as if doctors didn't want to fall ill and die. Charles Péguy, L'Argent, suiteFrench sociologist Pierre Bourdieu starts his 1988 book, Homo Academicus, with the above quote from poet and essayist Charles Péguy, commenting on the preference of the researcher to stand apart from what they research. Bourdieu's theoretical investigation of the academic world at a time of unrest and change (the 1968 University of Paris protests) presents to the reader the academy as an object for study. In doing so, Bourdieu brings the researcher into full display and argues that the authority and objectivity central to an academic's success is not inherent to the individual, but the result of the academic's position in the power structures of academia (Bourdieu, 1988).In writing his book, Bourdieu aimed to "exoticize the domestic", asking scholars to critically engage with the academic world they inhabit and to question what drives their research questions, methods, and conclusions. Was objective curiosity the sole driver of inquiry, or was research shaped by the influences of academic power and conformity? In order to promote radical change in academic standards and research, Bourdieu asked his peers to honestly reflect on their position of power, the production and consumption of knowledge, and their role in the validation of that knowledge (Forte, 2015).

There has been low confidence in the replicability and reproducibility of published psychological findings. Previous work has demonstrated that a population of psychologists exists that have used questionable research practices (QRPs), or behaviors during data collection, analysis, and publication that can increase the number of false-positive findings in the scientific literature. Across two survey studies, we sought to estimate the current size of the QRP-using population of American psychologists and to identify if this sub-population of scientists is stigmatized. Using a self-report direct estimator, we estimate approximately 18\% of American psychologists have used at least one QRP in the past 12 months. We then demonstrate the use of two additional estimators: the unmatched count estimate (an indirect self-report estimator) and the generalized network scale up method (an indirect social network estimator). Additionally, attitudes of psychologists towards QRP users, and ego network data collected from self-reported QRP users, suggest that QRP users are a stigmatized sub-population of psychologists. Together, these findings provide insight into how many psychologists are using questionable practices and how they exist in the social environment.