Machine-learning models have demonstrated great success in learning complex patterns that enable them to make predictions about unobserved data. In addition to using models for prediction, the ability to interpret what a model has learned is receiving an increasing amount of attention. However, this increased focus has led to considerable confusion about the notion of interpretability. In particular, it is unclear how the wide array of proposed interpretation methods are related and what common concepts can be used to evaluate them. We aim to address these concerns by defining interpretability in the context of machine learning and introducing the predictive, descriptive, relevant (PDR) framework for discussing interpretations. The PDR framework provides 3 overarching desiderata for evaluation: predictive accuracy, descriptive accuracy, and relevancy, with relevancy judged relative to a human audience. Moreover, to help manage the deluge of interpretation methods, we introduce a categorization of existing techniques into model-based and post hoc categories, with subgroups including sparsity, modularity, and simulatability. To demonstrate how practitioners can use the PDR framework to evaluate and understand interpretations, we provide numerous real-world examples. These examples highlight the often underappreciated role played by human audiences in discussions of interpretability. Finally, based on our framework, we discuss limitations of existing methods and directions for future work. We hope that this work will provide a common vocabulary that will make it easier for both practitioners and researchers to discuss and choose from the full range of interpretation methods.interpretability | machine learning | explainability | relevancy

No abstract

No abstract

No abstract

In terms of a single neuron’s long-distance communication, interpulse intervals (IPIs) are an attractive alternative to rate and binary codes. As a proxy for an IPI, a neuron’s time-to-spike can be found in the biophysical and experimental intracellular literature. Using the current, consensus layer V pyramidal neuron, the present study examines the feasibility of IPI-coding and examines the noise sources that limit the information rate of such an encoding. In descending order of importance, the noise sources are (i) synaptic variability, (ii) sodium channel shot-noise, followed by (iii) thermal noise. The biophysical simulations allow the calculation of mutual information, which is about 3.0 bits/spike. More importantly, while, by any conventional definition, the biophysical model is highly nonlinear, the underlying function that relates input intensity to the defined output variable is linear. When one assumes the perspective of a neuron coding via first hitting-time, this result justifies a pervasive and simplifying assumption of computational modelers—that a class of cortical neurons can be treated as linearly additive, computational devices.