A Comparison of the Percentage of “Yes” (Agree) Responses and Importance of Attributes (Constructs) Determined Using Check-All-That-Apply and Check-All-Statements (Yes/No) Question Formats in Five Countries

Abstract

Check All That Apply (CATA) has become a popular type of questionnaire response in sensory/consumer research in recent years. However, some authors have pointed out potential problems with the method. An online survey using either a Check-All-That-Apply (CATA) or Check-All-Statements (CAS) format for questions was conducted to provide a deeper understanding of the response data using the two question formats. With CATA, respondents select all terms or statements that apply from a given list, while, with CAS, respondents must respond (e.g., yes/no or agree/disagree) to each term or statement to show that it applies or does not apply. Respondents from five countries (Brazil, China, India, Spain, and the USA) were randomly assigned one of the two question formats (N = 200 per country per method). Motivations for eating items that belong to five food groups (starchy, protein, dairy, fruits, and desserts) were assessed. Results showed that CAS had higher percentages of “agree” responses than CATA. Also, the response ratio of CAS and CATA data was different, suggesting that interpretations of the data from each response type would also be different. Respondents in the USA, China, and Spain took longer to complete the CAS questionnaire, while respondents in Brazil and India had similar time durations for the two question formats. Overall, the CATA format was liked slightly more than the CAS format and fewer respondents dropped out of the survey when using the CATA response type. These findings suggest that the CATA format is quick and relatively easy for consumers to complete. However, it provokes fewer “apply” responses, which some psychologists suggest underestimates applicable terms or statements and CATA provides a different interpretation of data than the CAS format that requires consumers to respond to each term or statement. Further, CAS may overestimate the applicable terms. Consumer insights collected using CATA and CAS can lead to different decisions due to differences in data interpretation by researchers (e.g., marketers, nutritionists, product developers, and sensory scientists). More investigation is needed for the CATA and CAS question formats.

Comparing the Rate-All-That-Apply and Rate-All-Statements Question Formats across Five Countries

“…An eating motivation survey (EMS) which included questions on consumers’ motivations for eating or not eating food items that belong to five food groups was used for this online study [ 1 , 7 ]. The questionnaires were randomly assigned to respondents in either the RATA or RATING formats but not both (each respondent saw only one format).…”

“…The questionnaires were randomly assigned to respondents in either the RATA or RATING formats but not both (each respondent saw only one format). A total of 47 positive motivation terms that could be categorized into 16 eating motivation constructs were assessed in each question format of the EMS [ 1 , 7 ]. Each eating motivation construct consisted of three terms or subscales except for the choice limitation construct that had only two terms.…”

“…In quantitative consumer research, several question formats are used to collect respondents’ product descriptions and perceptions, opinions, beliefs and attitudes (POBAs). Questionnaires for consumer studies (e.g., online surveys, central location tests, and home-use tests) may include the highly popular Check-All-That-Apply (CATA) [ 1 , 2 , 3 ], or Check-All-Statements (CAS) [ 2 , 4 , 5 , 6 ], or RATING questions [ 7 , 8 , 9 , 10 ], or the Rate-All-That-Apply (RATA) questions [ 3 , 11 , 12 , 13 , 14 ].…”

“…Easy and non-tedious are the key reasons why CATA has gained popularity in recent years [ 2 , 15 , 16 , 17 , 18 ]. However, there is criticism of this question format because of the equivocal interpretations of the unchecked attributes on the listed options [ 1 , 4 , 5 , 19 ]. Conversely, the Check-All-Statements (CAS) or the forced-choice yes/no questions require respondents to provide a response (e.g., yes/no or agree/disagree) for each attribute or statement to show that it applies or does not apply.…”

“…Conversely, the Check-All-Statements (CAS) or the forced-choice yes/no questions require respondents to provide a response (e.g., yes/no or agree/disagree) for each attribute or statement to show that it applies or does not apply. Although CAS is immune to primacy bias (attributes at the top of the list are marked more frequently than those at the bottom of the list), which is prevalent with CATA questions, CAS has been associated with acquiescence bias, where respondents tend to mark or agree with the positive connotation for all survey questions [ 1 , 4 , 20 , 21 , 22 ].…”