An experiment comparing grids and item-by-item formats in web surveys completed through PCs and smartphones

Revilla, Mélanie; Toninelli, Daniele; Ochoa, Carlos

doi:10.1016/j.tele.2016.04.002

Cited by 23 publications

(36 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This study will also investigate whether the effect of the question format is moderated by the survey device and in particular mobile devices. Some studies have examined the matrix question among mobile respondents, and they show somewhat mixed results in completion time, item nonresponse, and response distribution (e.g., see Couper & Peterson, 2016;de Bruijne & Wijnant, 2013;Lattery, Bartolone, & Saunders, 2013;Peterson, Mechling, LaFrance, Swinehart, & Ham, 2013Revilla, Toninelli, & Ochoa, 2016. Given the smaller screen size of mobile devices compared to computers, responding to a matrix question may require more scrolling (both horizontal and vertical, depending on the size of the matrix), zooming, and pinching than for a item-by-item question and hence can be more challenging.…”

Section: The Current Studymentioning

confidence: 99%

Item-by-item Versus Matrix Questions

Liu

Cernat²

2016

Social Science Computer Review

View full text Add to dashboard Cite

While the choice of matrix versus item-by-item questions has received considerable attention in the literature, it is still unclear in what situation one is better than the other. Building upon the previous findings, this study expands this line of research by examining whether the difference between the two question types is moderated by the number of response options. Through a web survey experiment, this study compares matrix and item-by-item questions with 2, 3, 4, 5, 7, 9, and 11 response options. Additionally, we also investigate the impact of the device used to complete the survey on data quality. The results show that straight lining and response time are similar between the two question types across all response lengths, but item nonresponse tends to be higher for matrix than item-by-item question, especially among mobile respondents. Also measurement models reveal measurement equivalence between the two question types when there are fewer than seven response options. For matrices with 9 or 11 response options, analyses reveal substantial differences compared to item-by-item questions. Keywords matrix question, item-by-item question, web survey, survey experiment, data qualityAs more and more surveys are moving to online completion, be it by PC or mobile, survey researchers are striving to decrease the respondent burden without sacrificing data quality. One of the many important decisions in this sense is the use of grid questions. When asking multiple questions using the same set of response options, researchers often have two choices: either format them as item-byitem questions (asking them item by item) or group them into a matrix (also called grid) format and present them together. While the latter is more succinct, it can impact data quality (e.g., see Couper, Traugott, & Lamias, 2001).

show abstract

Section: The Current Studymentioning

confidence: 99%

Item-by-item Versus Matrix Questions

Liu

Cernat²

2016

Social Science Computer Review

View full text Add to dashboard Cite

show abstract

“…In this section, we synthesize in a very concise way the 10 main findings of this experiment. For further details about the experiment and/or the results, we refer to Revilla and Ochoa (2015), Toninelli and Revilla (2016) and Revilla et al (2017). The main findings are the following:…”

Section: The 10 Main Findingsmentioning

confidence: 87%

PCs versus Smartphones in answering web surveys: does the device make a difference?

Revilla

Toninelli

Ochoa³

2016

Surv Pract

Self Cite

View full text Add to dashboard Cite

“…Despite the compact form of item presentation, single-item measurement combined with the battery format of the question can undermine reliability of response (Revilla, Toninelli, and Ochoa 2017;Couper, Traugott, and Lamias 2001;Toepel and van Soest 2009). It may reduce the cognitive effort made by those responding (De Leeuw, Hox, and Dillman 2012) and increase the difficulty of answering for respondents who find it difficult to keep the instructions in mind, and pay less attention to the differences in specific items (Callegaro, Shand-Lubbers, and Dennis 2009), increasingly so with larger lists of items (Couper et al 2013).…”

Section: Battery Format Of Questionmentioning

confidence: 99%

Understanding the Moral Attitudes Scale in International Surveys: an Exploratory Study

Remizova

Rudnev

2019

Preprint

View full text Add to dashboard Cite

Several large international surveys, including the World Values Survey and the European Values Study, have been using the moral attitudes scale (MAS) to measure individual and country differences for decades. However, the validity of the instrument has been barely assessed. The current study addressed the concurrent and content validity of four popular MAS items (justifiability of homosexuality, suicide, prostitution, and euthanasia). A sample of 493 Russians completed both MAS and the four validated multi-item scales. Results demonstrated that, except for the homosexuality item, the MAS items had low concurrent validity, explaining less than 50% of the variance of the corresponding multi-item scales. The MAS items underestimated the justifiability of homosexuality, prostitution, and suicide, and overestimated the justifiability of euthanasia. The MAS homosexuality item appeared to be a precise measure of attitude towards male (but not female) homosexuality, responses to the prostitution and suicide items overlooked the positive arguments, and the euthanasia item tapped more into attitudes towards euthanasia of a dying person. The four items showed strong dependence on the overall justifiability. We conclude that separate MAS items should be used with caution, given the detected content bias and the items’ strong link to non-specific overall justifiability.

show abstract

An experiment comparing grids and item-by-item formats in web surveys completed through PCs and smartphones

Cited by 23 publications

References 19 publications

Item-by-item Versus Matrix Questions

Item-by-item Versus Matrix Questions

PCs versus Smartphones in answering web surveys: does the device make a difference?

Understanding the Moral Attitudes Scale in International Surveys: an Exploratory Study

Contact Info

Product

Resources

About