Are We Drawing the Right Conclusions? The Dangers of Answer Format Effects in Empirical Tourism Research

Abstract

Empirical tourism research has a long history and empirically based findings represent an important component of theory development and managerial insight. Nevertheless, empirical data of any kind is susceptible to misinterpretation. The aim of this study is to investigate to which extent empirical tourism research accounts for three sources of potential misinterpretation of results: (1) the occurrence of answer format effects, (2) the occurrence of culturally specific response styles, and (3) the selection of data analytic techniques appropriate for the data format. A review of 43 academic publications from 2000 and 2001 suggests that empirical tourism research is strongly guided by standards which have developed within the tourism research community and are not questioned anymore: ordinal answer formats dominate the field, ordinal data is analyzed using techniques requiring metric data and cross-cultural response styles are ignored, which is a particularly concerning finding given the amount of cross-cultural comparisons typically undertaken in tourism research. Recommendations for improvement are made. KeywordsKey words: answer format effects, response styles, cross-cultural research Nevertheless, empirical data of any kind is susceptible to misinterpretation. The aim of this study is to investigate to which extent empirical tourism research accounts for three sources of potential misinterpretation of results: (1) the occurrence of answer format effects, (2) the occurrence of culturally specific response styles, and (3) the selection of data analytic techniques appropriate for the data format. A review of 43 academic publications from 2000 and 2001 suggests that empirical tourism research is strongly guided by standards which have developed within the tourism research community and are not questioned anymore: ordinal answer formats dominate the field, ordinal data is analyzed using techniques requiring metric data and cross-cultural response styles are ignored, which is a particularly concerning finding given the amount of cross-cultural comparisons typically undertaken in tourism research. Recommendations for improvement are made.

‘Student experience’ surveys: some methodological considerations and an empirical investigation

“…Reviews of the literature by Smith (2004), van Herk, Poortinga, and Verhallen (2004) and Dolnicar (2006) show that African-American and Hispanic respondents, and respondents from some Mediterranean countries, tend to opt for categories at the extremes of scales, whereas respondents from Asia tend to avoid the extremes. Extremeness in response style tends to be positively correlated with age, and negatively with education level and household income.…”

Exploring the implications of cultural diversity for environmental learning at an African aquarium

“…Dolnicar and Grün (2007a), for example, noted that multiple methods of data collection should be used to decrease the possibility of data contamination by culture specific response styles (CSRS). Alternative strategies include binary responses (Yes or No), ranking scales or opposite word choices (Dolnicar, 2006;Harzing, 2006). In this study, the inclusion of the graphical INS scale provided an alternative measure, however, this scale contributed little to the final results.…”

Correcting for Response Style Effects on Service Quality Measures

“…Within tourism research, the topic was picked up only a few years ago (Dolnicar, 2007;Dolnicar & Grün, 2007a, 2007b. It is argued that some data collection methods are less prone to response styles than others, and that Likert-type scales tend to be the ones most affected by response styles (Smith & Reynolds, 2002).…”

“…Compared to the former approach, the latter causes additional concerns as to the reliability and validity of measurement outcomes. The particular problem of cross-cultural comparisons and potential differences thereof has received substantial attention in the field of consumer behavior research in general (Clarke, III, 2001;Greenleaf, 1992aGreenleaf, , 1992bvan Herk, Poortinga, & Verhallen, 2004) as well as, more recently, in tourism research (Dolnicar, 2007).…”