Dropout is a special case of the stochastic delta rule: faster and more accurate deep learning

Frazier-Logue, Noah; Hanson, Stephen José

doi:10.48550/arxiv.1808.03578

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dropout in a U-Net architecture can perform as a special case of the delta rule in which we introduce noise in the transmission of information [38] by randomly masking weights of the network. Dropout is presented as an especial case of delta rule called stochastic delta rule [39] in which each weight in the model is assigned as a random variable from a Gaussian distribution with the mean µ w ij and standard deviation of σ w ij [38]. Dropout, as an special case of stochastic delta rule, introduces a form of regularization that aids in escaping poor local minima.…”

Section: U-net With Dropoutmentioning

confidence: 99%

“…By randomly deactivating a subset of neurons during each training iteration, dropout prevents the network from relying too heavily on specific neurons or features. This selective deactivation encourages the remaining neurons to compensate and learn more robust representations, leading to a broader exploration of the weight space and increasing the odds of finding the optimum solution [38]. Additionally, keeping the dropout in the inference process will introduce stochasticity by generating results from a randomly selected sub-network and will result in an approximation of posterior distribution [40].…”

Section: U-net With Dropoutmentioning

confidence: 99%

See 1 more Smart Citation

Probabilistic Wildfire Segmentation Using Supervised Deep Generative Model from Satellite Imagery

et al. 2023

View full text Add to dashboard Cite

Wildfires are one of the major disasters among many and are responsible for more than 6 million acres burned in the United States alone every year. Accurate, insightful, and timely wildfire detection is needed to help authorities mitigate and prevent further destruction. Uncertainty quantification is always a crucial part of the detection of natural disasters, such as wildfires, and modeling products can be misinterpreted without proper uncertainty quantification. In this study, we propose a supervised deep generative machine-learning model that generates stochastic wildfire detection, allowing fast and comprehensive uncertainty quantification for individual and collective events. In the proposed approach, we also aim to address the patchy and discontinuous Moderate Resolution Imaging Spectroradiometer (MODIS) wildfire product by training the proposed model with MODIS raw and combined bands to detect fire. This approach allows us to generate diverse but plausible segmentations to represent the disagreements regarding the delineation of wildfire boundaries by subject matter experts. The proposed approach generates stochastic segmentation via two model streams in which one learns meaningful stochastic latent distributions, and the other learns the visual features. Two model branches join eventually to become a supervised stochastic image-to-image wildfire detection model. The model is compared to two baseline stochastic machine-learning models: (1) with permanent dropout in training and test phases and (2) with Stochastic ReLU activations. The visual and statistical metrics demonstrate better agreements between the ground truth and the proposed model segmentations. Furthermore, we used multiple scenarios to evaluate the model comprehension, and the proposed Probabilistic U-Net model demonstrates a better understanding of the underlying physical dynamics of wildfires compared to the baselines.

show abstract

Section: U-net With Dropoutmentioning

confidence: 99%

Section: U-net With Dropoutmentioning

confidence: 99%