Understanding from Deep Learning Models in Context

Abstract

This paper places into context how the term model in machine learning (ML) contrasts with traditional usages of scientific models for understanding and we show how direct analysis of an estimator's learned transformations (specifically, the hidden layers of a deep learning model) can improve understanding of the target phenomenon and reveal how the model organizes relevant information. Specifically, three modes of understanding will be identified, the difference between implementation irrelevance and functionally approximate irrelevance will be disambiguated, and how this distinction impacts potential understanding with these models will be explored.Additionally, by distinguishing between empirical link failures from representational ones, an ambiguity in the concept of link uncertainty will be addressed thus clarifying the role played by scientific background knowledge in enabling understanding with ML.

Do Machine Learning Models Represent Their Targets?

“…Since ML models engage in idealizations and can find patterns of interest in a way divorced from underlying real-world processes, like toy models, it seems like ML models do not actually represent their targets and that we should accept the ML representation hypothesis regarding their epistemic status. And indeed, in a recent paper, Tamir and Shech (2022) argue that ML models can fail to represent their targets, undermining their epistemic status. One example they highlight is the case of Esteva et al's (2017) melanoma classifier that reportedly does better at identifying melanoma compared to dermatologists.…”

“…This is an example of what I am calling a data processing idealization. Tamir and Shech (2022) suggest that the lack of similarity resulting from data processing idealizations can undermine how well ML models represent phenomena.…”