Diversity in Everyday Research Practice

Abstract

How should social science researchers deal with data inaccuracies? This article uses Web-based survey data collected from faculty members in three social science disciplines to document variation in views about data editing. Through an analysis of qualitative responses to a hypothetical vignette, the authors demonstrate that a wide range of opinion surrounds the “proper” use of data. Reactions are to some extent contingent on discipline and experience with different types of data and data collection methods. They also depend on characteristics of the data-editing situation—for example, whether the problem is with an independent or dependent variable. Even taking these social and situational factors into account, however, there is still substantial diversity in vignette responses. Normative standards that pervade other aspects of the research process have not yet emerged for data editing.

Scholarship and disciplinary practices

“…Ecologists' methods for locating key data, defined as that most critical to a research project, included inquiries made directly to museums, referrals from other scientists to survey data or databases, personal knowledge, and searches of peer-reviewed literature. Highlighting the work required to apply domain knowledge and experience to resolve ambiguities in data collection or analysis, Zimmerman's findings are consistent with Leahey et al (2003). Personal experiences in collecting data, along with the informal knowledge gained in fieldwork, helped to inform assessment practices involved in data re-use.…”

“…Leahey, Entwisle, and Einaudi (2003) documented general data editing rules or approaches across subdisciplinary groupings, following a Web-based survey of tenured faculty in sociology, psychology, and anthropology from U.S. research universities. Using a vignette scenario to ask qualitative questions in the proper use of data, they found variation related to discipline and experience with different types of data and data collection methods.…”

Serendipity: Towards a taxonomy and a theory

“…She shows large variation in how researchers deal with data, in how they clean datasets of apparently illogical, seemingly incorrect, or supposedly inaccurate data, and in researcher-opinions about 'proper' use of data. She finds that some of that variation depends on discipline, data collection method, characteristics of the data-editing situation like whether the problem is with an independent or dependent variable, and status and seniority (Leahey et al, 2003;Leahey, 2004). It seems reasonable to think that research governance structures might broadly and systematically affect 'what's in and what's omitted', and have important implications for serendipity measurement (Owen-Smith, 2001;Leahey, 2008).…”

Heterophily in rural Malawi

“…Consistent with this, the observer seldom asked how to code a particular piece of information or, more generally, how to resolve problems. Likewise, there were very few errors in the raw data (e.g., item non-response or missing codes), and, since data entry was conducted in the field, those few problems which did exist were caught early, making the data easy to 'clean' without having to make daring leaps of inference (Leahey et al 2003).…”