Innovating the Collection of Open-Ended Answers: The Linguistic and Content Characteristics of Written and Oral Answers to Political Attitude Questions

Abstract

The rapid increase in smartphone surveys and technological developments open novel opportunities for collecting survey answers. One of these opportunities is the use of open-ended questions with requests for oral instead of written answers, which may facilitate the answer process and result in more in-depth and unfiltered information. Whereas it is now possible to collect oral answers on smartphones, we still lack studies on the impact of this novel answer format on the characteristics of respondents' answers. In this study, we compare the linguistic and content characteristics of written versus oral answers to political attitude questions. For this purpose, we conducted an experiment in a smartphone survey (N = 2402) and randomly assigned respondents to an answer format (written or oral). Oral answers were collected via the open source 'SurveyVoice (SVoice)' tool, whereas written answers were typed in via the smartphone keypad. Applying length analysis, lexical structure analysis, sentiment analysis and structural topic models, our results reveal that written and oral answers differ substantially from each other in terms of lengths, structures, sentiments and topics. We find evidence thatThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Who Is Willing to Use Audio and Voice Inputs in Smartphone Surveys, and Why?

“…Particularly, the voice communication channel allows to collect “qualitative” answers in standardized web surveys. This combination of qualitative and quantitative methodologies enables researchers to gather unprecedented rich and nuanced information on respondents’ attitudes, behaviors, and beliefs by eliciting open narrations (Gavras & Höhne, 2020; Gavras et al, 2022). It also makes the communication process between researchers and respondents more natural by simulating daily conversation.…”

Improving and Analyzing Open-Ended Survey Responses

“…Schmidt et al (2020) recommend to select and combine different response features and text mining on theoretical considerations. Possible response features include response rates (Holland & Christian, 2009; Marshall, 2021; Zuell et al., 2015), answer lengths (Gavras et al, 2022; Holland & Christian, 2009; Mahon-Haft & Dillman, 2010; Schmidt et al, 2020), response and latency times (Schmidt et al, 2020; Smyth et al, 2009), answer tone (Gavras et al, 2022; Mahon-Haft & Dillman, 2010), number of mentioned topics (Gavras et al, 2022; Holland & Christian, 2009; Mahon-Haft & Dillman, 2010; Smyth et al, 2009), and interpretability or elaboration of answers (Schmidt et al, 2020; Smyth et al, 2009). For most features, different analysis approaches are possible (e.g., using character or word count for answer lengths, or different approaches to extract topics).…”

Assessment Based on Higher Order Thinking Skill on SKI Learning

“…Types of description questions are items that measure students' abilities in organizing, integrating, interpreting the tested learning materials, and expressing their answers in writing. (Gavras, K., Höhne, J. K., Blom, A., & Schoen, 2022;Walker, 2015). Explanation questions are useful for measuring student learning progress, intellectual quality, depth of understanding, and even learning difficulties experienced by students.…”