How (Not) to Solve the Problem: An Evaluation of Scholarly Responses to Common Source Bias

Abstract

Public administration scholars are beginning to pay more attention to the problem of common source bias, but little is known about the approaches that applied researchers are adopting as they attempt to confront the issue in their own research. In this essay, we consider the various responses taken by the authors of six articles in this journal. We draw attention to important nuances of the common measurement issue that have previously received little attention and run a set of empirical analyses in order to test the effectiveness of several proposed solutions to the common-source-bias problem. Our results indicate that none of the statistical remedies being used by public administration scholars appear to be reliable methods of countering the problem. Currently, it appears as though the only reliable solution is to find independent sources of data when perceptual survey measures are employed.

Sector Differences in the Public Service Motivation–Job Satisfaction Relationship: Exploring the Role of Organizational Characteristics

“…However, it is important to note here that common source bias is especially pronounced with respect to studies with perceptual measures, whereas our sector variable and also to some extent the organizational [22] characteristics could be argued to be more exogenous. Still, future studies would benefit from collecting data on individual-level measures and organizational characteristics from different data sources (Favero and Bullock, 2014, argue this to be the main way forward to rule out common source bias), and there should be continuous scholarly attention devoted to reducing social desirability bias in PSM measurement.…”

Cognitive styles, user acceptance and commitment to strategic plans in public organizations: an empirical analysis

“…This study relies on perceptual data collected through self-reported surveys which implies that common method bias (CMB) could be a concern (Favero and Bullock 2014). However, despite its limitations, using self-reported surveys as sole information source can be an appropriate measurement method when 'both the predictor and criterion variables are capturing an individual's perceptions, beliefs, judgments, or feelings' (Podsakoff, MacKenzie, and Podsakoff 2012, 549).…”

“…Second, our study employs cross-sectional data drawn from one selfreport survey. We are thus limited to associative relations and issues of common source bias cannot be completely ruled out (Favero and Bullock 2014). Third, although our sampling procedure was aimed at identifying expert informants, respondent bias could be an issue because we only survey individuals 'responsible' for plan formulation.…”

If Only We Knew What We Know: Factors for Mobilizing Citizen Participation in Community-based Emergency Preparedness

“…To date, researchers have understood that none of the statistical rectifications applied by public administration scholars appear to be reliable methods for solving the problem (Favero & Bullock, 2015). We acknowledge this point as a research limitation, and recommend that further researchers conduct an advanced analysis with the aim of developing more complete perceptual survey measures with independent data sources.…”