Cognitive interviewing to improve women's empowerment questions in surveys: Application to the health and nutrition and intrahousehold relationships modules for the project‐level Women's Empowerment in Agriculture Index

Abstract

In 2015, the United Nations adopted the Sustainable Development Goals, which include fostering gender equality and women's empowerment and ending hunger and malnutrition. To monitor progress and evaluate programmes that aim to achieve these goals, survey instruments are needed that can accurately assess related indicators. The project-level Women's Empowerment in Agriculture Index (pro-WEAI) is being developed to address the need for an instrument that is sensitive to changes in empowerment over the duration of an intervention. The pro-WEAI includes new modules with previously untested survey questions, including a health and nutrition module (focused on women's agency in this area) and an intrahousehold relationships module. This study uses cognitive interviewing to identify how new survey questions might be misinterpreted and to understand what experiences women are referencing when they respond to these questions. This was undertaken with the goal of informing revision to the modules. The study was conducted in Bangladesh with women from nuclear, extended, and migrant-sending households and from two regions of the country to identify difficulties with interpretation and response formulation across these groups. Findings revealed that questions were generally understood, but participants occasionally responded to the wrong part of the question, did not understand key phrases, or were uncomfortable with questions. The findings also suggested ways to revise the modules and strengthen the pro-WEAI. The revised pro-WEAI health and nutrition and intrahousehold relationships modules will advance the ability to measure changes in these domains and their relationship with the health and nutritional status of women and their children.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.Additional supporting information may be found online in the Supporting Information section at the end of the article. How to cite this article: Hannan A, Heckert J, James-Hawkins L, Yount KM. Cognitive interviewing to improve women's empowerment questions in surveys: Application to the health and nutrition and intrahousehold relationships modules for the project-level Women's Empowerment in Agriculture Index.

Realizing Inclusive SAI: contextualizing indicators to better evaluate gender and intergenerational inequity in SAI processes and outcomes – cases from Southern and Western Africa

“…Our study also contributes to emerging scholarship on improving survey questions on women's empowerment through cognitive interviewing (e.g. Hannan et al, 2019;Malapit et al, 2019).…”

“…Emerging scholarship on cognitive interviewing recognizes such challenges of making data collection on gender inequities more context appropriate. For instance, Hannan et al (2019) specifically target making the WEAI data-collection framework more participatory by changing the survey questions on women's empowerment. The WEAI is a major data-collection tool for measuring women's empowerment and inclusion in agriculture to aid decision-making in project/programme management and policymaking.…”

Using cognitive interviewing to bridge the intent‐interpretation gap for nutrition coverage survey questions in India

“…We also looked for patterns by child age subgroup and other sample characteristics (maternal age, education level, and birth order of the index child). Drawing on the typology of question failures (Scott et al, 2019 ) and mapping our data to previous literature (Hannan et al, 2020 ; Johnson, 2015 ), we used the cognitive stages as the basis of the analysis. In addition, facilitators or supportive question elements connected with better comprehension, retrieval and response were identified.…”

Exploring gendered experiences of time-use agency in Benin, Malawi, and Nigeria as a new concept to measure women’s empowerment

“…The team then developed a cognitive interview guide to evaluate the cognitive processes underlying responses to the proposed time-use agency survey questions and how well participants understand and respond to the proposed survey questions, and to inform potential improvements (Willis 2004;Willis and Miller 2011). These interviews were designed to identify potential discrepancies between researchers' and participants' understandings of a proposed survey question, participants' ability to recall the information requested, participants' motivation to respond (e.g., based on sensitivity of the question or social desirability), and participants' natural versus coded response options (Schuler, Lenzi, and Yount 2011;Hannan, Heckert, James-Hawkins, and Yount 2019;Qutteina, James-Hawkins, Al-Khelaifi, and Yount 2019). The findings from cognitive interviews at an early stage of module development can assist with question or module refinement to minimize sources of reporting error.…”

“…It also asked about specific activities and to what extent women and men make their own decisions about time spent on these activities, how often they talk to their spouse about each activity, and how often they quarrel with members of their household about the activities 1 . A cognitive interview guide was then developed for these survey questions, based on existing guides from studies in Bangladesh and Qatar (Schuler et al 2011;Hannan et al 2019;Qutteina et al 2019). The guide included a set of four scripted general probes for each survey item.…”