Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning

Abstract

Background Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. Methods A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. Results In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9–92.7), and specificity 75.6% (95% CI 73.1–78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2–91.5), but specificity increased to 85.1% (95%CI 82.6–87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200–200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69–0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66–0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. Conclusions The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.

Patient-level performance evaluation of a smartphone-based malaria diagnostic application

“…Most studies were only focusing on object-level or image-level evaluations so far. Even outside the scope of smartphone-based systems, only one such patient-level study [ 6 ] was found.…”

“…It can diagnose Plasmodium falciparum malaria by analysing a Giemsa-stained thick smear. Later, this prototype system evolved to a more advanced version and was renamed EasyScan Go [ 6 , 7 ], adding functions to diagnose non- P. falciparum species and an algorithm for thin smear analysis. Several other groups [ 8 – 10 ] also proposed systems with similar hardware designs.…”

“…EasyScan Go is taking the lead compared to others in that their algorithms can perform patient-level malaria diagnosis in all aspects, including parasite detection, quantitation, and species identification. They have evaluated their system based on the slide set that the World Health Organization (WHO) provides for external competence assessment of malaria microscopists and conducted field evaluation [ 6 , 7 ].…”

Emerging role of artificial intelligence in global health care

“…Computational models that accurately predict response to antiretroviral therapy have been developed and tested on cases of the disease from India and Southern Africa [6] . Machine learning has also been applied to the automated digital detection and quantification of malarial parasites, which could have a significant impact on the ability of laboratories in resource poor settings to diagnose and treat malaria [7] .…”

Digital microscopy and artificial intelligence could profoundly contribute to malaria diagnosis in elimination settings

“…Hence, most systems yet used thin films, significantly reducing the sensitivity of the method. Nevertheless, most published studies reached a close-to or similar sensitivity and specificity as expert microscopy (Das et al, 2022 ; Knapper et al, 2022 ).…”