Several recent studies of performance appraisal have used the "paper people" approach, in which raters read performance vignettes and then rate the performance of several hypothetical ratees. This approach may lead to systematically different outcomes from studies in which ratings are based on the direct or indirect (e.g., via videotape) observation of ratees' behavior. One hundred and eleven studies published between 1975 and 1984 were grouped into five major substantive categories, and a comparative meta-analysis was used to contrast the outcomes of paper people studies to those of similar studies in which ratings were based on the observation of ratee behavior. Effect sizes were found to be significantly larger in paper people studies, although this difference was not uniform across all research areas. Results are discussed in terms of differences in signal-to-noise ratios across the two methods.The authors wish to thank several colleagues who provided prepublication drafts of papers for our review.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.