A systematic review of the agreement of recall, home-based records, facility records, BCG scar, and serology for ascertaining vaccination status in low and middle-income countries

Abstract

Background: Household survey data are frequently used to estimate vaccination coverage - a key indicator for monitoring and guiding immunization programs - in low and middle-income countries. Surveys typically rely on documented evidence from home-based records (HBR) and/or maternal recall to determine a child’s vaccination history, and may also include health facility sources, BCG scars, and/or serological data. However, there is no gold standard source for vaccination history and the accuracy of existing sources has been called into question. Methods and Findings: We conducted a systematic review of literature published January 1, 1975 through December 11, 2017 that compared vaccination status at the child-level from at least two sources of vaccination history. 27 articles met inclusion criteria. The percentage point difference in coverage estimates varied substantially when comparing caregiver recall to HBRs (median: +1, range: -43 to +17), to health facility records (median: +5, range: -29 to +34) and to serology (median: -20, range: -32 to +2). Ranges were also wide comparing HBRs to facility-based records (median: +17, range: -61 to +21) and to serology (median: +2, range: -38 to +36). Across 10 studies comparing recall to HBRs, Kappa values exceeded 0.60 in 45% of comparisons; across 7 studies comparing recall to facility-based records, Kappa never reached 0.60. Agreement varied depending on study setting, coverage level, antigen type, number of doses, and child age. Conclusions: Recall and HBR provide relatively concordant vaccination histories in some settings, but both have poor agreement with facility-based records and serology. Long-term, improving clinical decision making and vaccination coverage estimates will depend on strengthening administrative systems and record keeping practices. Short-term, there must be greater recognition of imperfections across available vaccination history sources and explicit clarity regarding survey goals and the level of precision, potential biases, and associated resources needed to achieve these goals.

Knowledge, attitudes and practices on influenza vaccination during pregnancy in Quito, Ecuador

“…The vaccination rate reported in this study (36.6%) is lower than those reported for Ecuador in 2015 (55%) and 2016 (63%) and for those reported by other countries of the region, such as Bolivia (69%), Brazil (80%) and Argentina (100%) for 2016 [16]. However, the coverage could be underestimated due to the low percentage of women with perinatal or vaccination cards [28]. In Ecuador, the coverage of all vaccines (including the in uenza vaccine) has shown a gradual decrease since 2013 to 2016, and a slight increase from 2017 [15] .…”

Knowledge, attitudes and practices on influenza vaccine during pregnancy in Quito, Ecuador

“…The vaccination rate reported in this study (36.6%) is lower than those reported for Ecuador in 2015 (55%) and 2016 (63%) and for those reported by other countries of the region, such as Bolivia (69%), Brazil (80%) and Argentina (100%) for 2016 [16]. However, the coverage could be underestimated due to the low percentage of women with perinatal or vaccination card [28]. In Ecuador, the coverage of all vaccines (including the in uenza vaccine) indicates a gradual decrease since 2013 to 2016, and an slightly increase from 2017 [15] .…”

Implementing WHO guidance on conducting and analysing vaccination coverage cluster surveys: Two examples from Nigeria

“…Only just over half of respondents said the child had ever received a HBR, and only 29% could show the HBR to interviewers. Survey results for crude coverage therefore rely heavily on caretaker recall, which reduces data reliability [44][45][46] especially for the number of doses received in a multi-dose series-studies suggest that in some settings caretakers tend to under-report the number of doses received [44,46].…”