Precise language responses versus easy rating scales—Comparing respondents’ views with clinicians’ belief of the respondent’s views

Abstract

Background Closed-ended rating scales are the most used response format for researchers and clinicians to quantify mental states, whereas in natural contexts people communicate with natural language. The reason for using such scales is that they are typically argued to be more precise in measuring mental constructs; however, the respondents’ views as to what best communicates mental states are frequently ignored, which is important for making them comply with assessment. Methods We assessed respondents’ (N = 304) degree of depression using rating scales, descriptive words, selected words, and free text responses and probed the respondents for their preferences concerning the response formats across twelve dimensions related to the precision of communicating their mental states and the ease of responding. This was compared with the clinicians’ (N = 40) belief of the respondent’s view. Results Respondents found free text to be more precise (e.g., precision d’ = .88, elaboration d’ = 2.0) than rating scales, whereas rating scales were rated as easier to respond to (e.g., easier d’ = –.67, faster d’ = –1.13). Respondents preferred the free text responses to a greater degree than rating scales compared to clinicians’ belief of the respondents’ views. Conclusions These findings support previous studies concluding that future assessment of mental health can be aided by computational methods based on text data. Participants prefer an open response format as it allows them to elaborate, be precise, etc., with respect to their mental health issues, although rating scales are viewed as faster and easier.

Predicting attitudes toward ambiguity using natural language processing on free descriptions for open-ended question measurements

“…This survey method consisting of this free-text and NLP will allow for the measuring of an individual's personality in a more ecologically valid form; that is, an open-ended response method when expressing emotions and states in everyday life [8, 10,14]. In Kjell et al's study [8], questions were aimed to examine overall life satisfaction, such as "Overall, in your life, are you satis ed or not?…”

Question-based computational language approach outperforms rating scales in quantifying emotional states

“…We also selected some of the most common psychological concepts relating to emotional states, namely, depression, anxiety, satisfaction, and harmony in life, because they represent a blend of positive psychology and clinical psychology elements, each characterized by a specific emotional valence that can be quantified through the use of standardized rating scales and are typically highly correlated 30,32 . The selection of these four psychological constructs also aligns with previous works from our research group, ensuring consistency and facilitating the comparability of findings across studies 30,33,34 .…”

Predicting attitudes toward ambiguity using natural language processing on free descriptions for open-ended question measurements

“…This survey method consisting of free-text and NLP will allow for the measuring of an individual’s personality in a more ecologically valid form; that is, an open-ended response method when expressing emotions and states in everyday life 8 , 10 , 14 , 15 . In Kjell et al’s study 8 , questions aimed to examine overall life satisfaction, such as “Overall, in your life, are you satisfied or not?”; however, in this study, the question was constructed by specifying the situation and asking the respondent to imagine the situation, where “it can be interpreted in more than one way.” This allows the use of open-ended surveys that measure not only abstract concepts, such as life satisfaction, but also other personality traits and psychological states that are more specific.…”