Video and audio processing in paediatrics: a review

Abstract

Objective: Video and sound acquisition and processing technologies have seen great improvements in recent decades, with many applications in the biomedical area. The aim of this paper is to review the overall state of the art of advances within these topics in paediatrics and to evaluate their potential application for monitoring in the neonatal intensive care unit (NICU). Approach: For this purpose, more than 150 papers dealing with video and audio processing were reviewed. For both topics, clinical applications are described according to the considered cohorts—full-term newborns, infants and toddlers or preterm newborns. Then, processing methods are presented, in terms of data acquisition, feature extraction and characterization. Main results: The paper first focuses on the exploitation of video recordings; these began to be automatically processed in the 2000s and we show that they have mainly been used to characterize infant motion. Other applications, including respiration and heart rate estimation and facial analysis, are also presented. Audio processing is then reviewed, with a focus on the analysis of crying. The first studies in this field focused on induced-pain cries and the newest ones deal with spontaneous cries; the analyses are mainly based on frequency features. Then, some papers dealing with non-cry signals are also discussed. Significance: Finally, we show that even if recent improvements in digital video and signal processing allow for increased automation of processing, the context of the NICU makes a fully automated analysis of long recordings problematic. A few proposals for overcoming some of the limitations are given.

Machine Learning of Infant Spontaneous Movements for the Early Prediction of Cerebral Palsy: A Multi-Site Cohort Study

“…However, these risk assessments are either based on qualitative perception, requiring considerable training and clinical expertise (GMA), or demand highly expensive equipment (cerebral imaging) [8]. Thus, research on low-cost alternatives for early risk assessment of CP based on automatic and objective detection of infant spontaneous movements has rapidly increased the last two decades [9,10].…”

“…Automatic detection of infant spontaneous movements is based on several types of technology including 3D motion capture, inertial sensors, and video recordings [9]. The most clinically feasible technology is a video recording, which is non-intrusive, not dependent on body worn reflective markers or inertial sensors, and available in most clinical and home-based settings using commercially available video and smartphone cameras [10]. Because of the clinical use of observational GMA, large databases of video recordings and CP outcomes are becoming available.…”

Comparative Spectrographic Analysis of the Newborns’ Cry in the Presence of Tight Intrapartum Nuchal Cord vs. Normal Using the Neonat App. Preliminary Results

“…Thus, it is imperative that a non-invasive diagnosis, monitoring, or classification method should be assured for NBs in a risk category from their first hours of life. Thus, video and audio acquisitions have the advantage of proposing contactless and non-invasive ways to collect data for patients being cared for in hospital or at home [6].…”

“…Studies up to present concerning the evaluation of the NBs cry searched for abnormalities in the cries of newborns with multiple or severe problems during the neonatal period (low birth weight, respiratory disorders, jaundice, apnea, sudden infant death syndrome, deafness, hypoxia-based central nervous system (CNS) diseases, cleft palate, and asphyxia). Many other studies relied on pain-induced cries [6] because they offer a clearer image of the ability of the NB to signalize suffering. Furthermore, the cry was divided into language patterns as words, sentences, and phrases.…”

Preterm infants' limb-pose estimation from depth images using convolutional neural networks

“…Beside being time-consuming, this evaluation may be prone to inaccuracies due to clinicians' fatigue and susceptible to intra-and inter-clinician variability [7]. Some promising computer-assisted approaches have been proposed to support clinicians in detecting infants' movement from clinical devices (e.g., accelerometer, photopletismograph and force sensors) [8] and multimedia data (audio and video) [1], [9], [10]. With respect to intrusive clinical devices, RGB-D cameras can be easily integrated into standard clinical monitoring setup (e.g., over infants' cage) while not hindering infants' and health operators' movements.…”