Replication studies: A neglected aspect of psychological research.

Smith, Nathaniel C.

doi:10.1037/h0029774

Cited by 114 publications

(56 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Smith (183) and Dingman (56) direct their remarks to the reproduc tibility of research findings and argue for more replication, while Popham & Husek (159) argue fo r more use of criterion-referenced rather than norm· referenced measurement techniques.…”

Section: Methodological Developmentsmentioning

confidence: 96%

Personnel Selection

Bray¹,

Moses²

1972

Annu. Rev. Psychol.

View full text Add to dashboard Cite

The field of personnel selection has received a great deal of attention in the 3-year period covered by this review-January 1, 19681, , through Decem ber 31, 1970 In some respects it has been a period of discovery-the recent discovery, as noted by Campbell (30), of minority and disadvantaged groups has undoubtedly changed the complexion of the field. The specter of per sonnel selection by civil legislation has evoked considerable interest by parties both within and outside the profession. This in many ways has served as an impetus not only to discover new techniques but also to critically examine current practices concerning the full utilization of the disadvantaged, blacks, women, the elderly, and other special groups.At the same time, so me serious questions have been raised concerning personnel research strategies. A highly simplified, but rather typical, per sonnel selection research strategy frequently follows this format: Give job applicants an aptitude test of some sort, hire some of them, get ratings of job performance, correlate test scores with performance ratings, add more tests, use elaborate statistical methods, hopefully arriving at a degree of prediction which, in light of the selection ratio, is of some practical useful ness. The adequacy of this model, although vastly superior to the still all too common buy test-use test-no research approach, has in the past 3 years been questioned increasingly. The forces influencing the model in the direc tion of greater sophistication were already apparent in 1969, or even before, as noted by Owens & Jewell (155). They have intensified in the intervening years.Perhaps chief among these forces was the pressure, legal and otherwise, for fair employment practices. The demand for unbiased selection methods focused on testing and consequently placed personnel psychologists in an unaccusto med limelight. Such attention proved to be embarrassing, not only because testers had ignored minority groups but, as importantly, because the quality of research done even on the majority group left much to be de sired.Faced with renewed interest and pressures regarding selection practices, the general tenor of response has been disappointing. To some extent the 1 The helpful contrib utions of James R. Huck are gratefully acknowledged. , Articles appearing before and after this period are included where they pro vide continuity. 545 189 Annu. Rev. Psychol. 1972.23:545-576. Downloaded from www.annualreviews.org Access provided by University of California -San Francisco UCSF on 02/03/15. For personal use only. Quick links to online content Further ANNUAL REVIEWS

show abstract

Section: Methodological Developmentsmentioning

confidence: 96%

Personnel Selection

Bray¹,

Moses²

1972

Annu. Rev. Psychol.

View full text Add to dashboard Cite

show abstract

“…Systematic efforts at replication are indeed scarce within psychology (Schmidt, 2009;Smith, 1970;Tsang & Kwan, 1999). Given modern psychology's commitment to science, this scarcity seems puzzling.…”

mentioning

confidence: 99%

Rewarding Replications

Koole¹,

Lakens

2012

Perspect Psychol Sci

243

View full text Add to dashboard Cite

Although replications are vital to scientific progress, psychologists rarely engage in systematic replication efforts. In this article, we consider psychologists' narrative approach to scientific publications as an underlying reason for this neglect and propose an incentive structure for replications within psychology. First, researchers need accessible outlets for publishing replications. To accomplish this, psychology journals could publish replication reports in files that are electronically linked to reports of the original research. Second, replications should get cited. This can be achieved by cociting replications along with original research reports. Third, replications should become a valued collaborative effort. This can be realized by incorporating replications in teaching programs and by stimulating adversarial collaborations. The proposed incentive structure for replications can be developed in a relatively simple and cost-effective manner. By promoting replications, this incentive structure may greatly enhance the dependability of psychology's knowledge base.

show abstract

“…Replication is regarded as important in scientific investigation (cf. Smith, 1970), and appears essential in single-subject intensive research (Hersen & Barlow, 1976;Sidman, 1960). Hersen and Barlow (1976) recommended three types of replication procedures that are relevant to intensive research in clinical, school, and counseling psychology: direct replication, systematic replication, and clinical replication.…”

Section: In Applied Clinical Research Case-study Methods May (A) Fosmentioning

confidence: 97%

Chapter 2: Intensive Research: A Review of Methodological Issues in Clinical, Counseling, and School Psychology

Kratochwill

1979

Review of Research in Education

View full text Add to dashboard Cite

Clinical, school, and counseling psychologists are involved in implementing psychological principles and techniques toward the solution of applied problems. Those working with clients and various socialization agents (e.g., parents, teachers) who attempt to change behavior and the environment to promote human welfare should be guided by an empirical knowledge base that promotes the development of the most successful intervention program possible (Bergan, 1977; Goldman, 1978a Goldman, , 1978b. Moreover, these individuals are in a unique position to contribute to such a knowledge base. Both of these notions have remained cherished ideals, but both remain far from common practice in applied psychology (Kazdin & Marholin, 1978;Kazdin & Wilson, 1978;Ricks, 1976). Some doubt has been cast on the usefulness of various conventional research practices for applied psychology. At times counseling psychology has been its own hardest critic on research issues. Brayfield (1963) characterized the scientific contributions of counseling psychology as "unimpressive." Schmidt and Pepinsky (1965) reviewed counseling research and concluded that serious methodological problems pervaded the studies. Despite some positive comments on subsequent counseling research (Carkhuff, 1966; Myers, 1966), serious concerns raised over methodology have been more common (Hansen & Warner, 47 validity, the vast majority were one-shot case studies characterized by serious methodological flaws. Some writers have noted that poorly formulated research is not only worthless, but also deceptive (Whitely, 1967; Ripstra, Note 1). More recently, Goldman (1978a) raised a number of limitations in traditional research practices in the counseling field. His theme was also that the field needs more credible research strategies which are relevant to practice. (See also Sprinthall, 1975.) The related profession of school psychology has also had its share of concerns over applied research. Elsewhere, I have suggested (cf. Kratochwill, 1977) that school psychologists have long lamented that they are unable to perform research in applied settings (e.g., Bardon & Bennett, 1974;Eiserer, 1972;Guttentag, 1968;Itkin, 1967;Reger, 1965; Sciarra, 1968; White & Harris, 1961). Despite rhetoric about its desirability, historically research has not been a major role function of school psychologists (Eiserer, 1972). Many psychologist-practitioners are concerned with effecting change in their clients and are less concerned with formal research endeavors. Nevertheless, school psychologists are being increasingly called upon to document the effectiveness of their interventions; this is labeled "accountability". Moreover, there has been an upsurge of interest in their involvement in program evaluation research, as documented by various articles (e.g., Granger & Campbell, 1977) and an issue of the Journal of School Psychology devoted to recommending training and involvement in program evaluation research (e.g., Kratochwill & Bergan, 1978). Ysseldyke (1978) noted that most of the critical...

show abstract

Replication studies: A neglected aspect of psychological research.

Cited by 114 publications

References 17 publications

Personnel Selection

Personnel Selection

Rewarding Replications

Chapter 2: Intensive Research: A Review of Methodological Issues in Clinical, Counseling, and School Psychology

Contact Info

Product

Resources

About