The continuous monitoring of vital signs is a crucial aspect of medical care in neonatal intensive care units. Since cable-based sensors pose a potential risk for the immature skin of preterm infants, unobtrusive monitoring techniques using camera systems are increasingly investigated. The combination of deep learning–based algorithms and camera modalities such as RGB and infrared thermography can improve the development of cable-free methods for the extraction of vital parameters. In this study, a real-time approach for local extraction of temperatures on the body surface of neonates using a multi-modal clinical dataset was implemented. Therefore, a trained deep learning–based keypoint detector was used for body landmark prediction in RGB. Image registration was conducted to transfer the RGB points to the corresponding thermographic recordings. These landmarks were used to extract the body surface temperature in various regions to determine the central-peripheral temperature difference. A validation of the keypoint detector showed a mean average precision of 0.82. The registration resulted in mean absolute errors of 16.4 px (8.2 mm) for x and 22.4 px (11.2 mm) for y. The evaluation of the temperature extraction revealed a mean absolute error of 0.55 $$^{\circ }$$ ∘ C. A final performance of 31 fps was observed on the NVIDIA Jetson Xavier NX module, which proves real-time capability on an embedded GPU system. As a result, the approach can perform real-time temperature extraction on a low-cost GPU module. Graphical abstract

Background: Electrical Impedance Tomography (EIT) is a radiation-free technique for image reconstruction. However, as the inverse problem of EIT is non-linear and ill-posed, the reconstruction of sharp conductivity images poses a major problem. With the emergence of artificial neural networks (ANN), their application in EIT has recently gained interest. Methodology: We propose an ANN that can solve the inverse problem without the presence of a reference voltage. At the end of the ANN, we reused the dense layers multiple times, considering that the EIT exhibits rotational symmetries in a circular domain. To avoid bias in training data, the conductivity range used in the simulations was greater than expected in measurements. We also propose a new method that creates new data samples from existing training data. Results: We show that our ANN is more robust with respect to noise compared with the analytical Gauss–Newton approach. The reconstruction results for EIT phantom tank measurements are also clearer, as ringing artefacts are less pronounced. To evaluate the performance of the ANN under real-world conditions, we perform reconstructions on an experimental pig study with computed tomography for comparison. Conclusions: Our proposed ANN can reconstruct EIT images without the need of a reference voltage.

Background Monitoring the body temperature of premature infants is vital, as it allows optimal temperature control and may provide early warning signs for severe diseases such as sepsis. Thermography may be a non-contact and wireless alternative to state-of-the-art, cable-based methods. For monitoring use in clinical practice, automatic segmentation of the different body regions is necessary due to the movement of the infant. Methods This work presents and evaluates algorithms for automatic segmentation of infant body parts using deep learning methods. Based on a U-Net architecture, three neural networks were developed and compared. While the first two only used one imaging modality (visible light or thermography), the third applied a feature fusion of both. For training and evaluation, a dataset containing 600 visible light and 600 thermography images from 20 recordings of infants was created and manually labeled. In addition, we used transfer learning on publicly available datasets of adults in combination with data augmentation to improve the segmentation results. Results Individual optimization of the three deep learning models revealed that transfer learning and data augmentation improved segmentation regardless of the imaging modality. The fusion model achieved the best results during the final evaluation with a mean Intersection-over-Union (mIoU) of 0.85, closely followed by the RGB model. Only the thermography model achieved a lower accuracy (mIoU of 0.75). The results of the individual classes showed that all body parts were well-segmented, only the accuracy on the torso is inferior since the models struggle when only small areas of the skin are visible. Conclusion The presented multi-modal neural networks represent a new approach to the problem of infant body segmentation with limited available data. Robust results were obtained by applying feature fusion, cross-modality transfer learning and classical augmentation strategies.

Electrical impedance tomography (EIT) is an imaging modality that can estimate a visualization of the conductivity distribution inside the human body. However, the spatial resolution of EIT is limited because measurements are sensitive to noise. We investigate a technique to incorporate a priori information into the EIT reconstructions of the D-Bar algorithm. Our paper aims to help engineers understand the behavior of the D-Bar algorithm and its implementation. The a priori information is provided by a radar setup and a one-dimensional reconstruction of the radar data. The EIT reconstruction is carried out with a D-Bar algorithm. An intermediate step in the D-Bar algorithm is the scattering transform. The a priori information is added in this exact step to increase the spatial resolution of the reconstruction. As the D-Bar algorithm is widely used in the mathematical community and thus far has limited usage in the engineering domain, we also aim to explain the implementation of the algorithm and give an intuitive understanding where possible. Different parameters of the reconstruction algorithm are analyzed systematically with the help of the GREIT figures of merit. Even a limited one-dimensional a priori information can increase the reconstruction quality considerably. Artifacts from noisy EIT measurements are reduced. However, the selection of the amount of a priori information and the estimation of its value can worsen the reconstruction results again.

In today’s neonatal intensive care units, monitoring vital signs such as heart rate and respiration is fundamental for neonatal care. However, the attached sensors and electrodes restrict movement and can cause medical-adhesive-related skin injuries due to the immature skin of preterm infants, which may lead to serious complications. Thus, unobtrusive camera-based monitoring techniques in combination with image processing algorithms based on deep learning have the potential to allow cable-free vital signs measurements. Since the accuracy of deep-learning-based methods depends on the amount of training data, proper validation of the algorithms is difficult due to the limited image data of neonates. In order to enlarge such datasets, this study investigates the application of a conditional generative adversarial network for data augmentation by using edge detection frames from neonates to create RGB images. Different edge detection algorithms were used to validate the input images’ effect on the adversarial network’s generator. The state-of-the-art network architecture Pix2PixHD was adapted, and several hyperparameters were optimized. The quality of the generated RGB images was evaluated using a Mechanical Turk-like multistage survey conducted by 30 volunteers and the FID score. In a fake-only stage, 23% of the images were categorized as real. A direct comparison of generated and real (manually augmented) images revealed that 28% of the fake data were evaluated as more realistic. An FID score of 103.82 was achieved. Therefore, the conducted study shows promising results for the training and application of conditional generative adversarial networks to augment highly limited neonatal image datasets.