Introduction and Evaluation of an Electronic Tool for Improved Data Quality and Data Use during Malaria Case Management Supportive Supervision

Abstract

Although on-site supervision programs are implemented in many countries to assess and improve the quality of care, few publications have described the use of electronic tools during health facility supervision. The President's Malaria Initiative-funded MalariaCare project developed the MalariaCare Electronic Data System (EDS), a custom-built, opensource, Java-based, Android application that links to District Health Information Software 2, for data storage and visualization. The EDS was used during supervision visits at 4,951 health facilities across seven countries in Africa. The introduction of the EDS led to dramatic improvements in both completeness and timeliness of data on the quality of care provided for febrile patients. The EDS improved data completeness by 47 percentage points (42-89%) on average when compared with paper-based data collection. The average time from data submission to a final data analysis product dropped from over 5 months to 1 month. With more complete and timely data available, the Ministry of Health and the National Malaria Control Program (NMCP) staff could more effectively plan corrective actions and promptly allocate resources, ultimately leading to several improvements in the quality of malaria case management. Although government staff used supervision data during MalariaCare-supported lessons learned workshops to develop plans that led to improvements in quality of care, data use outside of these workshops has been limited. Additional efforts are required to institutionalize the use of supervision data within ministries of health and NMCPs.

Perspectives on Implementation Considerations and Costs of Malaria Case Management Supportive Supervision

“…To address these challenges later on, in all countries where large-scale OTSS was carried out (seven of nine countries), paper checklists were replaced with MalariaCare's electronic data system (EDS), an Android application that links to District Health Information System version 2 (DHIS2) (University of Oslo, Oslo, Norway) for data storage and visualization. 11 Supervisors using EDS were equipped with handheld tablets to enter and upload checklist data over a mobile network. In the first rounds of OTSS using EDS, 89% of health facilities (n = 1,556) had at least one complete clinical observation, compared with only 23% during the last set of visits using the paper checklist (n = 1,462).…”

“…For example, EDS was $17.78 more expensive per visit than paper checklists in one country where 120 facilities were visited but dropped to $0.16 more per visit in another country, where 782 facilities were visited. 11 For supportive supervision programs that are unable to implement electronic systems but still want to be able to use data for decision-making, extra emphasis during supervisor training on checklist completeness, plus follow-up with supervisors after data entry on key missing areas, should improve data quality. However, timeliness, completeness, and accuracy is not likely to reach the performance levels seen when using an electronic system.…”

“…Finally, using data from supportive supervision visits to make programmatic decisions also requires financial resources, whether for entering paper checklists in a database or using an electronic system, which may include the cost of tablets, airtime, and server space as compared with printing and data entry for paper checklists. The cost of both approaches is discussed in more detail by Burnett et al 11 Supplementing supportive supervision with other QA activities. Because the cost of supportive supervision prohibits the possibility of regularly visiting every facility, complementary or alternative methods of ensuring quality should be factored into an optimal QA system that includes supportive supervision.…”

Effect of Supportive Supervision on Competency of Febrile Clinical Case Management in Sub-Saharan Africa

“…Results gathered during observation of HCWs performing clinical consultations were either captured on a paper-based checklist and subsequently entered into a Microsoft Access database (Microsoft Corporation, Redmond, WA) or entered directly by supervisors into tablets running MalariaCare's electronic data system application which links to District Health Information System 2 (DHIS2) software (University of Oslo, Oslo, Norway) for data storage and analysis. 11 Results were analyzed at two levels: at the observation level (for HCW performance on individual checklist steps) and at the facility level (for overall facility improvement on clinical competencies over time). We reported performance on each of the individual 25 steps in the checklist as the proportion of HCWs observed who performed the step correctly during the first, second, and third visit among observations with no steps missing, among facilities with at least three visits and at least one complete observation at each time point.…”

Effect of Supportive Supervision on Malaria Microscopy Competencies in Sub-Saharan Africa

“…Results gathered during observation of on-duty health-care workers (HCWs) performing malaria microscopy on the day of an OTSS visit were either captured on a paper-based checklist and subsequently entered into a Microsoft Access database or entered directly by supervisors into MalariaCare's Electronic Data System (EDS), a system using District Health Information System version 2 (DHIS2) (Oslo, Norway) software to store and analyze data. 9 Tanzania, and Zambia. One country, Ghana, was excluded because the national laboratory authorities chose to use a different checklist; and another country, Madagascar, was excluded because intervention was limited to one city, Antananarivo, and thus too dissimilar to the other countries in the analysis.…”