A New Experiment on the use of Images to Answer Web Survey Questions

Abstract

Images might provide richer and more objective information than text answers to open-ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open-ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between-subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open-ended questions with text; (2) a treatment group answering open-ended questions with images; and (3) another treatment group answering open-ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants'This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Protocol for a web survey experiment studying the feasibility of asking respondents to capture and submit photos of the books they have at home and the resulting data quality

“…As for the respondents’ evaluation with surveys asking for images, the literature is also scarce. Bosch et al (2022) found that between 12.4% and 20.0% of respondents liked answering with images, while 47.3% to 52.7% liked answering similar questions in conventional ways. Further, nearly 80% of respondents found it easy to answer in conventional ways, versus around 50% for answering by sharing images, which also resulted taking more time for participants.…”

“…The participation rates range from 10% in the study by Jäckle et al (2019) asking for photos of receipts, to 55% in the study by Bosch et al (2019a) when asking for a photo of the place where respondents are when answering the survey. Moreover, greater item nonresponse has been found when respondents had to answer by sharing images than when using conventional response formats ( Bosch et al , 2022 ; Ilic et al , 2022 ).…”

Pictures Instead of Survey Questions: An Experimental Investigation of the Feasibility of Using Pictures in a Housing Survey

“…In another study, Bosch et al (forthcoming) used an opt‐in panel to experiment with three different answer formats to survey questions: a text response, uploading an image, or uploading an image after receiving a motivational message. Their results show that the image conditions resulted in much lower response rates, most likely due to the increased burden, or situational aspects when the image questions are asked.…”