Machines learn to infer stellar parameters just by looking at a large number of spectra

Abstract

Machine learning has been widely applied to clearly defined problems of astronomy and astrophysics. However, deep learning and its conceptual differences to classical machine learning have been largely overlooked in these fields. The broad hypothesis behind our work is that letting the abundant real astrophysical data speak for itself, with minimal supervision and no labels, can reveal interesting patterns that may facilitate discovery of novel physical relationships. Here, as the first step, we seek to interpret the representations a deep convolutional neural network chooses to learn, and find correlations in them with current physical understanding. We train an encoder–decoder architecture on the self-supervised auxiliary task of reconstruction to allow it to learn general representations without bias towards any specific task. By exerting weak disentanglement at the information bottleneck of the network, we implicitly enforce interpretability in the learned features. We develop two independent statistical and information-theoretical methods for finding the number of learned informative features, as well as measuring their true correlation with astrophysical validation labels. As a case study, we apply this method to a data set of ∼270 000 stellar spectra, each of which comprising ∼300 000 dimensions. We find that the network clearly assigns specific nodes to estimate (notions of) parameters such as radial velocity and effective temperature without being asked to do so, all in a completely physics-agnostic process. This supports the first part of our hypothesis. Moreover, we find with high confidence that there are ∼4 more independently informative dimensions that do not show a direct correlation with our validation parameters, presenting potential room for future studies.

Investigation of stellar magnetic activity using variational autoencoder based on low-resolution spectroscopic survey

“…The variational autoencoder (VAE; Kingma & Welling 2014) is a variant of AE, and is widely used in astronomy (e.g. Portillo et al 2020;Sedaghat et al 2021). Although the architecture of VAE is similar to that of AE, the concept behind is very different, and VAE is based on the variational Bayesian inference (Kingma & Welling 2014).…”

A robust estimator of mutual information for deep learning interpretability

“…for backpropagation when training DL models), such neural-network-based estimators do not necessarily return an accurate estimate of equation (1), are heavily dependent on the training hyperparameters, and have been shown to suffer from a poor variance-bias tradeoff [75]. The use of MI estimates for interpreting deep representation learning has recently been investigated as well [19,32,79,80]; however, exploiting MI to interpret deep representation learning requires a robust density estimate of the joint probability distribution between latent variables and relevant physical parameters, and the uncertainties on the MI estimate to be quantified, ensuring that any trends in MI are statistically significant.…”

Discovering the building blocks of dark matter halo density profiles with neural networks

“…Most of these methods require prior knowledge of the system of interest, for example a priori knowledge of the relevant variables or the underlying dimensionality. Recently, Sedaghat et al [33] adopted a similar architecture to SciNet in an unsupervised setting, where a neural network is trained to find a lowdimensional representation of stellar spectra. Similar to our work, they used mutual information for interpretability; however, their use of mutual information was limited to identifying potential correlations between the latent representation and previously known parameters.…”