Object Detection During Newborn Resuscitation Activities

Abstract

Birth asphyxia is a major newborn mortality problem in low-resource countries. International guideline provides treatment recommendations; however, the importance and effect of the different treatments are not fully explored. The available data is collected in Tanzania, during newborn resuscitation, for analysis of the resuscitation activities and the response of the newborn. An important step in the analysis is to create activity timelines of the episodes, where activities include ventilation, suction, stimulation etc. Methods:The available recordings are noisy real-world videos with large variations. We propose a two-step process in order to detect activities possibly overlapping in time. The first step is to detect and track the relevant objects, like bag-mask resuscitator, heart rate sensors etc., and the second step is to use this information to recognize the resuscitation activities. The topic of this paper is the first step, and the object detection and tracking are based on convolutional neural networks followed by post processing. Results: The performance of the object detection during activities were 96.97 % (ventilations), 100 % (attaching/removing heart rate sensor) and 75 % (suction) on a test set of 20 videos. The system also estimate the number of health care providers present with a performance of 71.16 %. Conclusion:The proposed object detection and tracking system provides promising results in noisy newborn resuscitation videos. Significance: This is the first step in a thorough analysis of newborn resuscitation episodes, which could provide important insight about the importance and effect of different newborn resuscitation activities.

Signal Processing for Newborn Survival from labour to resuscitation

“…An alternative approach for automatically annotating newborn resuscitation episodes is by using deep leaning on videos overlooking the resuscitation table. One such system, ORAA-net [86], has previously been proposed by our research team. The system consists of four main steps: 1) Object detection, 2) Region proposal, 3) Activity recognition, and 4) generation of Activity timelines.…”

“…The system consists of four main steps: 1) Object detection, 2) Region proposal, 3) Activity recognition, and 4) generation of Activity timelines. The first step is based on a deep learning system to detect objects such as bag-mask ventilator, suction devices, and health care hands [86]. Regions around these objects are proposed and used in a new network to recognize stimulation, ventilation, suction, and if the newborn is covered or uncovered [87].…”

“…In a future scenario we might have NeoBeat available in some hospitals, video signals in other hospitals, ventilation data from some hospitals, or several of these components. We are planning to fuse the output of the NBstim algorithm with the output of the automated video analysis [86], and potentially output from ventilator signals [84] to produce more reliable timelines of activities, also including ventilation and suction when possible.…”

“…A number of sensor data have been collected during newborn resuscitation at partner hospitals in Tanzania during the research project; pressure and flow from the bag-mask resuscitator (BMR), dry-electrode electrocardiogram (ECG) signals and signals from an accelerometer using a prototype of the NeoBeat 6 , attached over the abdomen of the newborn. The detection of bag-mask ventilation can relatively easy be performed using the flow and pressure signals from sensors mounted in the BMR [84], and detection and recognition of treatment activities during newborn resuscitation using deep neural networks on videos of resuscitation [86] has been described in earlier work from this research group. Videos of the resuscitation is often not available, or the view can be blocked by some of the activities.…”

Automatic identification of stimulation activities during newborn resuscitation using ECG and accelerometer signals

“…There might as well be potential for real time decision support during resuscitation. A number of sensor data have been collected during The detection of bag-mask ventilation can relatively easy be performed using the flow and pressure signals from sensors mounted in the BMR [8], and detection and recognition of treatment activities during newborn resuscitation using deep neural networks on videos of resuscitation [9] has been described in earlier work from this research group. Videos of the resuscitation is often not available, or the view can be blocked by some of the activities.…”