Measurement agreement of the self-administered questionnaire of the Belgian Health Interview Survey: Paper-and-pencil versus web-based mode

Abstract

Before organizing mixed-mode data collection for the self-administered questionnaire of the Belgian Health Interview Survey, measurement effects between the paper-and-pencil and the web-based questionnaire were evaluated. A two-period cross-over study was organized with a sample of 149 employees of two Belgian research institutes (age range 22–62 years, 72% female). Measurement agreement was assessed for a diverse range of health indicators related to general health, mental and psychosocial health, health behaviors and prevention with kappa coefficients and intraclass correlation (ICC). The quality of the data collected by both modes was evaluated by quantifying the missing, ‘don’t know’ and inconsistent values and data entry mistakes. Good to very good agreement was found for all categorical indicators with kappa coefficients superior to 0.60, except for two mental and psychosocial health indicators namely the presence of a sleeping disorder and of a depressive disorder (kappa≥0.50). For the continuous indicators high to acceptable agreement was observed with ICC superior to 0.70. Inconsistent answers and data-entry mistakes were only occurring in the paper-and-pencil mode. There were no less missing values in the web-based mode compared to the paper-and-pencil mode. The study supports the idea that web-based modes provide, in general, equal responses to paper-and-pencil modes. However, health indicators based upon factual and objective items tend to have higher measurement agreement than indicators requiring an assessment of personal subjective feelings. A web-based mode greatly facilitates the data-entry process and guides the completing of a questionnaire. However, item non-response was not positively affected.

Acceptability, internal consistency and test–retest reliability of scales to assess parental and nursery staff’s self-efficacy, motivation and knowledge in relation to pre-school children’s nutrition, oral health and physical activity

“…This is important to consider in low-to middle-income countries where device and Internet access may not be available to administer tablet/web-based forms of the questionnaire. However, there is evidence to suggest that acceptability, internal consistency and test-retest reliability outcomes are comparable between paper-based and device/web-based forms of questionnaire administration (34)(35)(36) . Due to the limitations stated above, caution needs to be taken when interpreting the magnitude of the results and deciding whether to remove certain items for use in studies.…”

‘Only God can promise healing.’: help-seeking intentions and lay beliefs about cures for post-traumatic stress disorder among Sub-Saharan African asylum seekers in Germany

“…Asylum seekers preferred the paper-and-pencil version, as they favoured the personal contact with researchers and expressed difficulties with their internet connection. However, past research supports the idea that webbased data collection provides, in general, equal responses as paper-and-pencil modes (Braekman et al, 2018). For the German participants without a migration background the online survey (n = 99) was promoted by the researchers through mailing lists, social media, and through personal contacts via word-of-mouth recommendation.…”

Comparing the efficiency of paper-based and electronic data capture during face-to-face interviews

“…This is an issue not often faced by other studies trialling EDC, but one that is significant to fisheries sampling. In other disciplines, where data is collected by individuals going door-to-door and seldom collecting biological data, EDC improved both the timeliness and efficiency of data collection [ 20 , 21 ]. The real test for EDC in on-site fisheries sampling is its adaptability to difficult environmental conditions, an issue easily mitigated with additional staff but the potential for increased errors may need to be considered for surveys that are only completed by a single field officer.…”

“…Previous studies have compared EDC with PDC although the focus has been on the efficiency of devices rather than the accuracy of the data collected [ 19 ]. [ 20 ] explored the application of EDC in health surveys comparing efficiencies of PDC compared to EDC when used by respondents completing a questionnaire. [ 21 , 22 ] also demonstrated how EDC can be used to complete face-to-face interviews comparing the use of tablets and notebooks to PDC, ultimately concluding that EDC was preferred by survey staff, saved time, and reduced survey costs.…”