Cognitive Interviewing to Improve Questionnaires for Justice-Involved Youth

Silva, Jason R.; Fera, Beth; Sudula, Susruta; Koetzle, Deborah; Schwalbe, Craig S.

doi:10.1177/0306624x19839597

Cited by 4 publications

(11 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the cognitive processing framework is applicable to adult respondents, it is especially useful for conceptualizing specific issues related to the response process for children and adolescents as they have different cognitive capacities than adults (Silva et al, 2019). According to Beatty and Willis (2007), age and cognitive functioning are the most important factors that contribute to a respondent's ability to accurately interpret and respond to an item.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The meaning of items can vary as a function of age and across developmental levels, with younger respondents having more limited cognitive processing and more difficulty with interpretation, vocabulary, reading, and abstract concepts (Koskey et al, 2010, Woolley et al, 2004. The extent to which survey questions can be read, comprehended, and answered by respondents of a certain age has been described in the literature as developmental validity (Silva et al, 2019;Woolley et al, 2004) or cognitive validity (Karabenick et al, 2007). Tourangeau's (1984) model provides a framework for the cognitive processes used in the survey response process, including the developmental cognitive processing considerations specific to children and adolescents.…”

Section: Literature Reviewmentioning

confidence: 99%

“…or 6) specific, such as "Why would someone worry about getting the flu?" While evaluators have used a combination of both think-aloud and verbal probing techniques with adults (Domanska et al, 2018;Pepper et al, 2018;Vreeman et al, 2014), and youth (Koskey et al, 2010;Mulcahey et al, 2009;Silva et al, 2019), the emphasis is more frequently on verbal probing (Willis & Artino, 2013). Verbal probing is an especially useful technique with children and adolescents for exploring abstract concepts or context specific items that may be difficult for them to describe (Silva et al, 2019).…”

Section: Cognitive Interviewing Techniquesmentioning

confidence: 99%

“…While cognitive interviewing is most commonly used with adults, they have also been completed with children as young as 8 years old (Irwin et al, 2009;Koskey et al, 2010;Mulcahey et al, 2009;Rebok et al, 2001), adolescents (Irwin & Stafford, 2016;Joffer et al, 2016;Koskey et al, 2010;Norris et al, 2014;Pepper et al, 2018;Silva et al, 2019), and young adults (Hinds et al, 2016;Karpen & Hagemeier, 2016;Kramer & Schwartz, 2017). Although younger participants struggle with comprehension more than older children, they are able to understand items, response options, directions, and can identify problems with difficult language (Irwin et al, 2009;Rebok et al, 2001).…”

Section: Sample Selectionmentioning

confidence: 99%

“…Cognitive interviewing techniques are used to assess the cognitive processes utilized by participants to respond to items. This approach is particularly appropriate for testing items with children and adolescents who have more limited cognitive capacities than adults and vary in their cognitive development (Silva et al, 2019). In addition, children and adolescents have a unique perspective on their life experiences and context (Katz et al, 2017), making it essential to explore how they interpret items that have been developed or adapted for use with them in their context.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Using Cognitive Interviewing to Test Youth Survey and Interview Items in Evaluation: A Case Example

Urban

Linver

2020

J Multidiscip Eval

View full text Add to dashboard Cite

Background: Cognitive interviewing is a pretesting tool used by evaluators to increase item and response option validity. Cognitive interviewing techniques are used to assess the cognitive processes utilized by participants to respond to items. This approach is particularly appropriate for testing items with children and adolescents who have more limited cognitive capacities than adults, vary in their cognitive development, and have a unique perspective on their life experiences and context. Purpose: This paper presents a case example of cognitive interviewing with youth as part of a national program evaluation, and aims to expand the use of cognitive interviewing as a pretesting tool for both quantitative and qualitative items in evaluation studies involving youth. Setting: Youth participants were located in four regions of the United States: Northeast, Central, Southern, and Western. Interviewers were located at Montclair State University. Intervention: Not applicable. Research design: A cognitive interview measure was designed to include a subset of survey items, interview questions, and verbal probes, to evaluate if these items and questions would be understood as intended by both younger and older youth participants. An iterative design was used with cognitive interviewing testing rounds, analysis, and revisions. Data Collection and Analysis: The cognitive interview was administered by phone to 10 male youth, five from the 10-13-year-old age range and five from the 15-17-year-old age range. Interviews were audio-recorded, transcribed, reviewed, and coded. Survey items and interview questions were revised based on feedback from the participants and consensus agreement among the evaluation team. Item revisions were included in further testing rounds with new participants. Findings: As a result of using cognitive interviewing to pretest survey and interview items with youth, response errors were identified. Participants did not understand some of the items and response options as intended, indicating problems with validity. These findings support the use of cognitive interviewing for testing and modifying survey items adapted for use with youth, as well as qualitative interview items. Additionally, the perspective of the youth participants was valuable for informing decisions to modify items and helping the evaluators learn the participants’ program culture and experiences. Based on the findings and limitations of the study, we give practice recommendations for future studies using cognitive interviewing with a youth sample. Keywords: cognitive interviewing; item validity; response error; verbal probes; pre-testing surveys; qualitative evaluation; interviewing children and adolescents; survey development

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Cognitive Interviewing Techniquesmentioning

confidence: 99%

Section: Sample Selectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Using Cognitive Interviewing to Test Youth Survey and Interview Items in Evaluation: A Case Example

Urban

Linver

2020

J Multidiscip Eval

View full text Add to dashboard Cite

show abstract

Initial Validity Evidence for the Youth Interpersonal Behaviors Questionnaire in Sport (Y-IBQ)

Wekesser

Erickson

Bateman

et al. 2023

Measurement in Physical Education and Exercise Science

View full text Add to dashboard Cite

Adolescents’ Understanding of Smoking and Vaping Risk Language: Cognitive Interviews to Inform Scale Development

Kurtzman

Sheldon

Adams

et al. 2022

Nicotine &Amp; Tobacco Research

View full text Add to dashboard Cite

Introduction Perceived message effectiveness (PME) is a common metric to understand receptivity to tobacco prevention messages, yet most measures have been developed with adults. We examined adolescents’ interpretation of language within candidate items for a new youth-targeted PME measure using cognitive interviewing. We sought to understand the meaning adolescents’ assigned to our candidate PME items to improve item wording. Methods Participants were 20 adolescents ages 13-17 from the United States. Cognitive interviews used a structured guide to elicit feedback on comprehension, answer retrieval, and language regarding a set of Reasoned Action Approach-based survey items that assessed the PME of smoking and vaping prevention ads. We employed thematic analysis to synthesize findings from the interviews. Results Interviews identified three main issues related to survey items: ambiguity of language, word choice (risk and other terminology), and survey item phrasing. Adolescents preferred direct, definitive language over more ambiguous phrasing which they saw as less serious (e.g., “will” instead of “could”). For risk terminology, they preferred terms such as “harmful” and “dangerous” over “risky,” which was viewed as easy to discount. The term “negative effects” was interpreted as encompassing a broader set of tobacco harms than “health effects.” Adolescents said that the term “vape” was preferable to “e-cigarette,” and identified ways to simplify item wording for greater clarity. Conclusions Tobacco risk terms that appear similar differ in meaning to adolescents, and more direct and unambiguous language is preferred. Our findings informed changes to the PME scale items to improve clarity and reduce measurement error. Implications This study adds to the literature on how adolescents interpret tobacco prevention language. Adolescents may interpret terminology differently than adults, which could lead to ambiguity in meaning and thus measurement error. Through cognitive interviewing, we identified and improved the language in a youth-focused perceived message effectiveness (PME) measure for tobacco and vaping prevention.

show abstract

Cognitive Interviewing to Improve Questionnaires for Justice-Involved Youth

Cited by 4 publications

References 52 publications

Using Cognitive Interviewing to Test Youth Survey and Interview Items in Evaluation: A Case Example

Using Cognitive Interviewing to Test Youth Survey and Interview Items in Evaluation: A Case Example

Initial Validity Evidence for the Youth Interpersonal Behaviors Questionnaire in Sport (Y-IBQ)

Adolescents’ Understanding of Smoking and Vaping Risk Language: Cognitive Interviews to Inform Scale Development

Contact Info

Product

Resources

About