Activity Recognition From Newborn Resuscitation Videos

Abstract

Birth asphyxia is one of the leading causes of neonatal deaths. A key for survival is performing immediate and continuous quality newborn resuscitation. A dataset of recorded signals during newborn resuscitation, including videos, has been collected in Haydom, Tanzania, and the aim is to analyze the treatment and its effect on the newborn outcome. An important step is to generate timelines of relevant resuscitation activities, including ventilation, stimulation, suction, etc., during the resuscitation episodes. Methods: We propose a two-step deep neural network system, ORAA-net, utilizing low-quality video recordings of resuscitation episodes to do activity recognition during newborn resuscitation. The first step is to detect and track relevant objects using Convolutional Neural Networks (CNN) and post-processing, and the second step is to analyze the proposed activity regions from step 1 to do activity recognition using 3D CNNs. Results: The system recognized the activities newborn uncovered, stimulation, ventilation and suction with a mean precision of 77.67 %, a mean recall of 77,64 %, and a mean accuracy of 92.40 %. Moreover, the accuracy of the estimated number of Health Care Providers (HCPs) present during the resuscitation episodes was 68.32 %. Conclusion: The results indicate that the proposed CNN-based two-step ORAAnet could be used for object detection and activity recognition in noisy low-quality newborn resuscitation videos. Significance: A thorough analysis of the effect the different resuscitation activities have on the newborn outcome could potentially allow us to optimize treatment guidelines, training, debriefing, and local quality improvement in newborn resuscitation.

NewbornTime - improved Newborn Care based on Video and Artificial Intelligence - Study protocol

“…The data collection also includes the preparation of an existing VL video dataset of newborn resuscitation from Haydom Lutheran hospital in Tanzania (481 VL videos) [30] collected during the Safer Births study [27], and from SUS (237 VL videos) collected as part of the NeoBeat study [32]. The Safer Births and NeoBeat studies are ongoing and new data will also be included.…”

Resuscitation of term and near-term newborns in low-resourced settings: Studies of positive end-expiratory pressure and expired CO2 during bag-mask ventilation at birth

“…Papers published by the group range from epidemiological studies and studies to evaluate cost effectiveness of interventions 192,[225][226][227][228][229] via studies of machine learning for automatic interpretation of data [230][231][232] to studies to describe practise, 196 evaluate effect of training methodologies, [233][234][235][236] studies to improve understanding of the basic physiology of newborn transition and resuscitation 74,134,138,221,237 and RCTs to compare new equipment to existing standards. 238,239…”

Signal Processing for Newborn Survival from labour to resuscitation

“…The first step is based on a deep learning system to detect objects such as bag-mask ventilator, suction devices, and health care hands [86]. Regions around these objects are proposed and used in a new network to recognize stimulation, ventilation, suction, and if the newborn is covered or uncovered [87]. The recognized activities are finally used to create timelines describing the resuscitation episode.…”