Role of Layers and Neurons in Deep Learning With the Rectified Linear Unit

Takekawa, Akira; Kajiura, Masayuki; Fukuda, Hiroya

doi:10.7759/cureus.18866

Cited by 10 publications

(8 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The encoder (Additional file 1 : Figure S1) is characterized by a series of convolutional and pooling layers that doubles the size of the feature map while reducing the number of channels by half. Each convolution uses a 3 × 3 kernel and it is followed by batch normalization and ReLU activation function [ 16 , 17 ]. The decoder (Additional file 1 : Figure S2) recovers the spatial information back to the image space through a series of upsampling and convolution operations, thus increasing the output resolution.…”

Section: Methodsmentioning

confidence: 99%

Cardiovascular magnetic resonance images with susceptibility artifacts: artificial intelligence with spatial-attention for ventricular volumes and mass assessment

Penso

Babbaro

Moccia

et al. 2022

Journal of Cardiovascular Magnetic Resonance

View full text Add to dashboard Cite

Background Segmentation of cardiovascular magnetic resonance (CMR) images is an essential step for evaluating dimensional and functional ventricular parameters as ejection fraction (EF) but may be limited by artifacts, which represent the major challenge to automatically derive clinical information. The aim of this study is to investigate the accuracy of a deep learning (DL) approach for automatic segmentation of cardiac structures from CMR images characterized by magnetic susceptibility artifact in patient with cardiac implanted electronic devices (CIED). Methods In this retrospective study, 230 patients (100 with CIED) who underwent clinically indicated CMR were used to developed and test a DL model. A novel convolutional neural network was proposed to extract the left ventricle (LV) and right (RV) ventricle endocardium and LV epicardium. In order to perform a successful segmentation, it is important the network learns to identify salient image regions even during local magnetic field inhomogeneities. The proposed network takes advantage from a spatial attention module to selectively process the most relevant information and focus on the structures of interest. To improve segmentation, especially for images with artifacts, multiple loss functions were minimized in unison. Segmentation results were assessed against manual tracings and commercial CMR analysis software cvi42(Circle Cardiovascular Imaging, Calgary, Alberta, Canada). An external dataset of 56 patients with CIED was used to assess model generalizability. Results In the internal datasets, on image with artifacts, the median Dice coefficients for end-diastolic LV cavity, LV myocardium and RV cavity, were 0.93, 0.77 and 0.87 and 0.91, 0.82, and 0.83 in end-systole, respectively. The proposed method reached higher segmentation accuracy than commercial software, with performance comparable to expert inter-observer variability (bias ± 95%LoA): LVEF 1 ± 8% vs 3 ± 9%, RVEF − 2 ± 15% vs 3 ± 21%. In the external cohort, EF well correlated with manual tracing (intraclass correlation coefficient: LVEF 0.98, RVEF 0.93). The automatic approach was significant faster than manual segmentation in providing cardiac parameters (approximately 1.5 s vs 450 s). Conclusions Experimental results show that the proposed method reached promising performance in cardiac segmentation from CMR images with susceptibility artifacts and alleviates time consuming expert physician contour segmentation.

show abstract

Section: Methodsmentioning

confidence: 99%

Cardiovascular magnetic resonance images with susceptibility artifacts: artificial intelligence with spatial-attention for ventricular volumes and mass assessment

Penso

Babbaro

Moccia

et al. 2022

Journal of Cardiovascular Magnetic Resonance

View full text Add to dashboard Cite

show abstract

“…, respectively, the residual unit function  is composed of a combination of rectified linear unit (ReLU) function [38] and Conv (as illustrated in Figure 9 below), and…”

Section: Residual Unitmentioning

confidence: 99%

“…where the input and output of the k th residual unit are denoted by X (k+1) and X (k+2) , respectively, the residual unit function ϕ is composed of a combination of rectified linear unit (ReLU) function [38] and Conv (as illustrated in Figure 9 below), and θ (k) represents all the parameters of the k th residual unit.…”

Section: Residual Unitmentioning

confidence: 99%

Prediction of Inbound and Outbound Passenger Flow in Urban Rail Transit Based on Spatio-Temporal Attention Residual Network

Yang,

Dong,

Yang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Passenger flow prediction is a critical approach to ensure the effective functioning of urban rail transit. However, there are few studies that combine multiple influencing factors for short-term passenger flow prediction. It is also a challenge to accurately predict passenger flow at all stations in the line at the same time. To overcome the above limitations, a deep learning-based method named ST-RANet is proposed, which consists of three spatio-temporal modules and one external module. The model is capable of predicting inbound and outbound passenger flow for all stations within the network simultaneously. We model the spatio-temporal data in terms of three temporal characteristics, including closeness, period, and trend. For each characteristic, we construct a spatio-temporal module that innovatively integrates the attention mechanisms into the middle of residual units and convolutional neural networks (CNNs) to extract and learn spatio-temporal features. Subsequently, the results of the three modules are integrated using a parameter matrix method, which allows for dynamic aggregation based on data. The integration results are further combined with external factors, such as holidays and meteorological information, to obtain passenger flow prediction values for each station. The proposed model is validated using real data from Beijing Subway, and optimized parameters are applied for 30-min granularity passenger flow predictions. Comparing the performance against 5 baseline models and verifying with data from multiple lines, the results indicate that the proposed ST-RANet model shows the best results. It is demonstrated that the method proposed in this paper has high prediction accuracy and good applicability.

show abstract

“…As shown in Figure 6, the activation function used for the neurons of each layer is the ReLU [46], but the last one has a linear activation function since the task addressed is a regression task. The activation function φ is the decision-making element that defines the decision boundary in the input space by setting a threshold.…”

Section: Dense Neural Network Modelmentioning

confidence: 99%

The Use of Artificial Intelligence Approaches for Performance Improvement of Low-Cost Integrated Navigation Systems

Alteriis

Ruggiero

Prete

et al. 2023

Sensors

View full text Add to dashboard Cite

In this paper, the authors investigate the possibility of applying artificial intelligence algorithms to the outputs of a low-cost Kalman filter-based navigation solution in order to achieve performance similar to that of high-end MEMS inertial sensors. To further improve the results of the prototype and simultaneously lighten filter requirements, different AI models are compared in this paper to determine their performance in terms of complexity and accuracy. By overcoming some known limitations (e.g., sensitivity on the dimension of input data from inertial sensors) and starting from Kalman filter applications (whose raw noise parameter estimates were obtained from a simple analysis of sensor specifications), such a solution presents an intermediate behavior compared to the current state of the art. It allows the exploitation of the power of AI models. Different Neural Network models have been taken into account and compared in terms of measurement accuracy and a number of model parameters; in particular, Dense, 1-Dimension Convolutional, and Long Short Term Memory Neural networks. As can be excepted, the higher the NN complexity, the higher the measurement accuracy; the models’ performance has been assessed by means of the root-mean-square error (RMSE) between the target and predicted values of all the navigation parameters.

show abstract

Role of Layers and Neurons in Deep Learning With the Rectified Linear Unit

Cited by 10 publications

References 4 publications

Cardiovascular magnetic resonance images with susceptibility artifacts: artificial intelligence with spatial-attention for ventricular volumes and mass assessment

Cardiovascular magnetic resonance images with susceptibility artifacts: artificial intelligence with spatial-attention for ventricular volumes and mass assessment

Prediction of Inbound and Outbound Passenger Flow in Urban Rail Transit Based on Spatio-Temporal Attention Residual Network

The Use of Artificial Intelligence Approaches for Performance Improvement of Low-Cost Integrated Navigation Systems

Contact Info

Product

Resources

About