Automated versus physician assignment of cause of death for verbal autopsies: randomized trial of 9374 deaths in 117 villages in India

Abstract

Background Verbal autopsies with physician assignment of cause of death (COD) are commonly used in settings where medical certification of deaths is uncommon. It remains unanswered if automated algorithms can replace physician assignment. Methods We randomized verbal autopsy interviews for deaths in 117 villages in rural India to either physician or automated COD assignment. Twenty-four trained lay (non-medical) surveyors applied the allocated method using a laptop-based electronic system. Two of 25 physicians were allocated randomly to independently code the deaths in the physician assignment arm. Six algorithms (Naïve Bayes Classifier (NBC), King-Lu, InSilicoVA, InSilicoVA-NT, InterVA-4, and SmartVA) coded each death in the automated arm. The primary outcome was concordance with the COD distribution in the standard physician-assigned arm. Four thousand six hundred fifty-one (4651) deaths were allocated to physician (standard), and 4723 to automated arms. Results The two arms were nearly identical in demographics and key symptom patterns. The average concordances of automated algorithms with the standard were 62%, 56%, and 59% for adult, child, and neonatal deaths, respectively. Automated algorithms showed inconsistent results, even for causes that are relatively easy to identify such as road traffic injuries. Automated algorithms underestimated the number of cancer and suicide deaths in adults and overestimated other injuries in adults and children. Across all ages, average weighted concordance with the standard was 62% (range 79–45%) with the best to worst ranking automated algorithms being InterVA-4, InSilicoVA-NT, InSilicoVA, SmartVA, NBC, and King-Lu. Individual-level sensitivity for causes of adult deaths in the automated arm was low between the algorithms but high between two independent physicians in the physician arm. Conclusions While desirable, automated algorithms require further development and rigorous evaluation. Lay reporting of deaths paired with physician COD assignment of verbal autopsies, despite some limitations, remains a practicable method to document the patterns of mortality reliably for unattended deaths. Trial registration ClinicalTrials.gov , NCT02810366. Submitted on 11 April 2016. Electronic supplementary material The online version of this article (10.1186/s12916-019-1353-2) contains supplementary material, which is available to authorized users.

Causes and circumstances of maternal death: a secondary analysis of the Community-Level Interventions for Pre-eclampsia (CLIP) trials cohort

“…15 A second publication evaluated the causes of all deaths in adults and children, comparing physicianassigned cause of death at tertiary hospitals involved in final care with InterVA4 and InterVA5, which found high concordance (eg, InterVA5 correlation coefficient of 0•86 [0•79-0•93] for adults). 6 InterVA4 has been compared with physician review for deaths in general [16][17][18][19] and for maternal death specifically. 20 In Africa and Asia among adults or the general population, concordance between physician review and InterVA4 has been variably reported, as high (≥80%) 17,18 or not.…”

“…6 InterVA4 has been compared with physician review for deaths in general [16][17][18][19] and for maternal death specifically. 20 In Africa and Asia among adults or the general population, concordance between physician review and InterVA4 has been variably reported, as high (≥80%) 17,18 or not. 16,19,21 Among 86 maternal deaths analysed at the healthfacility level in Malawi, fair to good agreement was reported between an expert physician review panel and InterVA4 for ICDMM categories 1-7 (Cohen's κ=0•66), although the expert physician review panel assigned cause of death as unspecified more often than both InterVA4 and InterVA5.…”

National Burden Estimates of healthy life lost in India, 2017: an analysis using direct mortality data and indirect disability data

“…We applied chance-corrected mortality fraction accuracy to calculate the population-level concordance between the NBE and GBD, taking into account chance agreement. 20 100% concordance would mean identical cause of death distribution in the two comparisons. The major source of uncertainty in the NBE does not arise from random errors: the sample size for the MDS is very large and completeness of the sources of vital rates is high, as evaluated independently by the UN.…”

“…17 Earlier evaluations of the MDS have shown high comparability with relevant hospital or clinical data, strong reproducibility of the dual physician-coded verbal autopsies, and generally low rates of misclassification in children and young and middle-age adults. 15,16,20 Moreover, the uncertainty in diagnosis on verbal autopsy is not likely to affect the relative ranking of diseases.…”

Automated verbal autopsy: from research to routine use in civil registration and vital statistics systems

“…Yet, to improve population health for these (mostly) rural communities where most deaths occur at home, efficient mortality reduction strategies require reliable COD information systems, and the only viable means to do so is the routine use of automated VA. It is simply not imaginable that requiring busy rural physicians to do so on a regular basis is sustainable, despite claims to the contrary [36]. Our experience under the D4H Initiative in more than a dozen countries suggests that reliable methods such as SmartVA can readily, cost-effectively and in a timely fashion yield the essential information needed to guide public policy [14,15].…”