Statistical significance in psychological research.

Abstract

Most theories in the areas of personality, clinical, and social psychology predict no more than the direction of a correlation, group difference, or treatment effect. Since the null hypothesis is never strictly true, such predictions have about a SO-SO chance of being confirmed by experiment when the theory in question is false, since the statistical significance of the result is a function of the sample size. Confirmation of a single directional prediction should usually add little to one's confidence in the theory being tested. Most theories should be tested by multiple corroboration and most empirical generalizations by constructive replication. Statistical significance is perhaps the least important attribute of a good experiment; it is never a sufficient condition for claiming that a theory has been usefully corroborated, that a meaningful empirical fact has been established, or that an experimental report ought to be published.

Scientific Utopia

“…In some domains, requiring replication is not an onerous requirement because data collection is easy. But in other domains, it is difficult or even impossible to conduct a direct replication because of the resource demands or unique opportunities for data collection (Elms, 1975;Lykken, 1968). If replication were essential for every new phenomenon, then researchers might be disinclined to pursue new and challenging ideas to ensure publishability of what they produce.…”

Externalizing in preschoolers and early adolescents: A cross-study replication of a family model.

“…A construct replication (Lykken, 1968) is a test of the crossvalidation of findings. Such validation increases our confidence that the relationships found between the constructs can be used to formulate more general theories of family adaptation.…”

Prosocial response to client-instigated victimization: The roles of forgiveness and workgroup conflict.

“…Using a daily diary study, we are able to extend our curvilinear model to workplace outcomes as in Study 2, yet, with this design, we are also able to look at our questions longitudinally and understand how much individuals in our sample tend to change over time (i.e., within-person effects). Additionally, as a constructive replication, Study 3 extends the generalizability of the previous research by not exactly duplicating the previous methods (Lykken, 1968). We use a service employee sample outside of human service work.…”