Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers

Binder, Alexander; Montavon, Grégoire; Bach, Sebastian; Müller, Klaus‐Robert; Samek, Wojciech

doi:10.48550/arxiv.1604.00825

Cited by 14 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, the widespread study and application of saliency methods speaks to their intuitive appeal(Springenberg et al, 2015;Selvaraju et al, 2017;Fong and Vedaldi, 2017;Baehrens et al, 2010;Simonyan et al, 2014;Zeiler and Fergus, 2014;Shrikumar et al, 2017;Sundararajan et al, 2017;Smilkov et al, 2017;Dabkowski and Gal, 2017;Ancona et al, 2018;Yamashita et al, 2018;Rajpurkar et al, 2017;Wang et al, 2017Wang et al, , 2020Puri et al, 2019;Mott et al, 2019;Wang et al, 2016b;Iyer et al, 2018;Greydanus et al, 2018;Nikulin et al, 2019).But recent studies have cast doubt on their reliability(Alqaraawi et al, 2020;Kindermans et al, 2017;Sundararajan et al, 2017;Binder et al, 2016;Shrikumar et al, 2017;Chandrasekaran et al, 2017;Adebayo et al, 2018;Wang et al, 2019;Atrey et al, 2019) Sundararajan et al (2017)Binder et al (2016);Shrikumar et al (2017) identified that many studies of saliency methods lack a clear baseline for comparison (i.e. null3 Researcher degrees of freedom are described bySimmons et al (2011) as follows: "In the course of collecting and analyzing data, researchers have many decisions to make: Should more data be collected?…”

mentioning

confidence: 99%

Towards falsifiable interpretability research

Leavitt,

Morcos

2020

Preprint

View full text Add to dashboard Cite

Methods for understanding the decisions of and mechanisms underlying deep neural networks (DNNs) typically rely on building intuition by emphasizing sensory or semantic features of individual examples. For instance, methods aim to visualize the components of an input which are "important" to a network's decision, or to measure the semantic properties of single neurons. Here, we argue that interpretability research suffers from an over-reliance on intuition-based approaches that risk-and in some cases have caused-illusory progress and misleading conclusions. We identify a set of limitations that we argue impede meaningful progress in interpretability research, and examine two popular classes of interpretability methods-saliency and single-neuron-based approaches-that serve as case studies for how overreliance on intuition and lack of falsifiability can undermine interpretability research. To address these concerns, we propose a strategy to address these impediments in the form of a framework for strongly falsifiable interpretability research. We encourage researchers to use their intuitions as a starting point to develop and test clear, falsifiable hypotheses, and hope that our framework yields robust, evidence-based interpretability methods that generate meaningful advances in our understanding of DNNs. * Equal contribution † Work performed as part of the Facebook AI Residency program

show abstract

mentioning

confidence: 99%

Towards falsifiable interpretability research

Leavitt,

Morcos

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Lastly, the discriminative end-to-end nature of many deeplearning approaches (including our own) makes these models difficult to interpret, often hindering their ability to be useful for understanding the underlying physical theory. There are numerous definitions of what it means for deep learning to be interpretable, as well as numerous proposed methods for making some of these interpretations (e.g., Simonyan et al 2013;Yosinski et al 2015;Binder et al 2016;Montavon et al 2017). Specifically, for our case of predicting stellar properties from time-series data, it would be insightful to know what features of the light curves contributed most to the prediction of a particular stellar property.…”

Section: Discussionmentioning

confidence: 99%

Data-driven Derivation of Stellar Properties from Photometric Time Series Data Using Convolutional Neural Networks

Blancato

Ness²,

Huber³

et al. 2022

ApJ

View full text Add to dashboard Cite

Stellar variability is driven by a multitude of internal physical processes that depend on fundamental stellar properties. These properties are our bridge to reconciling stellar observations with stellar physics and to understand the distribution of stellar populations within the context of galaxy formation. Numerous ongoing and upcoming missions are charting brightness fluctuations of stars over time, which encode information about physical processes such as the rotation period, evolutionary state (such as effective temperature and surface gravity), and mass (via asteroseismic parameters). Here, we explore how well we can predict these stellar properties, across different evolutionary states, using only photometric time-series data. To do this, we implement a convolutional neural network, and with data-driven modeling we predict stellar properties from light curves of various baselines and cadences. Based on a single quarter of Kepler data, we recover the stellar properties, including the surface gravity for red giant stars (with an uncertainty of ≲0.06 dex) and rotation period for main-sequence stars (with an uncertainty of ≲5.2 days, and unbiased from ≈5 to 40 days). Shortening the Kepler data to a 27 days Transiting Exoplanet Survey Satellite–like baseline, we recover the stellar properties with a small decrease in precision, ∼0.07 for log g and ∼5.5 days for P rot, unbiased from ≈5 to 35 days. Our flexible data-driven approach leverages the full information content of the data, requires minimal or no feature engineering, and can be generalized to other surveys and data sets. This has the potential to provide stellar property estimates for many millions of stars in current and future surveys.

show abstract

“…There are numerous definitions of what it means for deep learning to be interpretable, as well as numerous proposed methods for making some of these interpretations (e.g. Simonyan et al 2013;Yosinski et al 2015;Binder et al 2016;Montavon et al 2017). Specifically, for our case of predicting stellar properties from time series data, it would be insightful to know what features of the light curves contributed most to the prediction of a particular stellar property.…”

Section: Discussionmentioning

confidence: 99%

Data-driven derivation of stellar properties from photometric time series data using convolutional neural networks

Blancato¹,

Ness²,

Huber³

et al. 2020

Preprint

View full text Add to dashboard Cite

Stellar variability is driven by a multitude of internal physical processes that depend on fundamental stellar properties. These properties are our bridge to reconciling stellar observations with stellar physics, and for understanding the distribution of stellar populations within the context of galaxy formation. Numerous ongoing and upcoming missions are charting brightness fluctuations of stars over time, which encode information about physical processes such as rotation period, evolutionary state (such as effective temperature and surface gravity), and mass (via asteroseismic parameters). Here, we explore how well we can predict these stellar properties, across different evolutionary states, using only photometric time series data. To do this, we implement a convolutional neural network, and with data-driven modeling we predict stellar properties from light curves of various baselines and cadences. Based on a single quarter of Kepler data, we recover stellar properties, including surface gravity for red giant stars (with an uncertainty of 0.06 dex), and rotation period for main sequence stars (with an uncertainty of 5.2 days, and unbiased from ≈5 to 40 days). Shortening the Kepler data to a 27-day TESS-like baseline, we recover stellar properties with a small decrease in precision, ∼0.07 dex for log g and ∼5.5 days for P rot , unbiased from ≈5 to 35 days. Our flexible data-driven approach leverages the full information content of the data, requires minimal feature engineering, and can be generalized to other surveys and datasets. This has the potential to provide stellar property estimates for many millions of stars in current and future surveys.

show abstract

Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers

Cited by 14 publications

References 0 publications

Towards falsifiable interpretability research

Towards falsifiable interpretability research

Data-driven Derivation of Stellar Properties from Photometric Time Series Data Using Convolutional Neural Networks

Data-driven derivation of stellar properties from photometric time series data using convolutional neural networks

Contact Info

Product

Resources

About