Recall accuracy of weekly automated surveys of health care utilization and infectious disease symptoms among infants over the first year of life

Abstract

Automated surveys, by interactive voice response (IVR) or email, are increasingly used for clinical research. Although convenient and inexpensive, they have uncertain validity. We sought to assess the accuracy of longitudinally-collected automated survey responses compared to medical records. Using data collected from a well-characterized, prospective birth cohort over the first year of life, we examined concordance between guardians’ reports of their infants’ health care visits ascertained by weekly automated survey (IVR or email) and those identified by medical chart review. Among 180 survey-visit pairs, concordance was 51%, with no change as number of visits per baby increased. Accuracy of recall was higher by email compared to IVR (61 vs. 43%; adjusted OR = 2.5 95% CI: 1.3–4.8), did not vary by health care encounter type (hospitalization: 50%, ER: 64%, urgent care: 44%, primary care: 52%; p = 0.75), but was higher for fever (77%, adjusted OR = 5.1 95%CI: 1.5–17.7) and respiratory illness (58%, adjusted OR = 2.9 95%CI: 1.5–5.8) than for other diagnoses. For the 75 mothers in these encounters, 69% recalled at least one visit; among 41 mothers with two or more visits, 85% recalled at least one visit. Predictors of accurate reporting by mothers after adjusting for illness in the baby included increased age and increased years of education (age per year, β = 0.05, p = 0.03; education per year, β = 0.08, p = 0.04). Additional strategies beyond use of automated surveys are needed to ascertain accurate health care utilization in longitudinal cohort studies, particularly in healthy populations with little motivation for accurate reporting.

Real Time Monitoring of Respiratory Viral Infections in Cohort Studies Using a Smartphone App

“…Another major benefit of a smartphone app is the ability to pre-program logic to define an event and automate "take-a-sample" alerts, rather than relying on participant education and recall for sampling 14,16,17 . This enables standardised study criteria, while reducing missed sampling of true events or unnecessary additional sampling, particularly in cases where health literacy may be limited 31 .…”

“…Automated alerts through text-messages or emails have also been increasingly used to prompt study activities 17,30 , but a dedicated smartphone app is necessary to pair reminders with data collection/monitoring. Another major benefit of a smartphone app is the ability to pre-program logic to define an event and automate “take-a-sample” alerts, rather than relying on participant education and recall for sampling 14,16,17 . This enables standardised study criteria, while reducing missed sampling of true events or unnecessary additional sampling, particularly in cases where health literacy may be limited 31 .…”

“…Significant respiratory illness events can be identified by monitoring hospital and/or primary healthcare attendance 13 , but this approach works best in regions with centralised healthcare and will miss relevant illnesses not requiring medical attention. Questionnaires and interviews can retrospectively capture illness events but are subject to recall bias particularly when performed infrequently 14,15 . Another approach is to use regular study visits and/or baseline virus sampling to capture illnesses 12 , with more frequent interactions decreasing the likelihood of missing events but increasing resource utilisation and participant burden.…”

“…Finally, symptom diaries have been frequently utilised for either retrospective identification of illness episodes or as a tool to promote reporting of events 8 . However, traditional paperbased symptom diaries are often onerous on participants and/or can be inconsistently completed 14,[16][17][18] . Novel approaches to enable robust illness detection with reduced participant and resource burden are still needed.…”