Determining spectroscopic redshifts by using<i>k</i>nearest neighbor regression

Kügler, S. D.; Polsterer, Kai Lars; Hoecker, Maximilian

doi:10.1051/0004-6361/201424801

Cited by 13 publications

(13 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The disadvantages are that it is computationally intensive for large datasets, given that all pairwise distances have to be calculated; further, the results are highly dependent on the exact training set used, making the results fairly inconsistent especially for small training sets. An example of a recent application of KNN in astronomy can be found in Kügler et al (2015) where it is applied to spectroscopic redshift estimation.…”

Section: K-nearest Neighborsmentioning

confidence: 99%

Photometric Supernova Classification With Machine Learning

Lochner

McEwen

Peiris

et al. 2016

ApJS

209

255

View full text Add to dashboard Cite

Automated photometric supernova classification has become an active area of research in recent years in light of current and upcoming imaging surveys such as the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope, given that spectroscopic confirmation of type for all supernovae discovered will be impossible. Here, we develop a multi-faceted classification pipeline, combining existing and new approaches. Our pipeline consists of two stages: extracting descriptive features from the light curves and classification using a machine learning algorithm. Our feature extraction methods vary from model-dependent techniques, namely SALT2 fits, to more independent techniques fitting parametric models to curves, to a completely model-independent wavelet approach. We cover a range of representative machine learning algorithms, including naive Bayes, k -nearest neighbors, support vector machines, artificial neural networks and boosted decision trees (BDTs). We test the pipeline on simulated multi-band DES light curves from the Supernova Photometric Classification Challenge. Using the commonly used area under the curve (AUC) of the Receiver Operating Characteristic as a metric, we find that the SALT2 fits and the wavelet approach, with the BDTs algorithm, each achieves an AUC of 0.98, where 1 represents perfect classification. We find that a representative training set is essential for good classification, whatever the feature set or algorithm, with implications for spectroscopic follow-up. Importantly, we find that by using either the SALT2 or the wavelet feature sets with a BDT algorithm, accurate classification is possible purely from light curve data, without the need for any redshift information.

show abstract

Section: K-nearest Neighborsmentioning

confidence: 99%

Photometric Supernova Classification With Machine Learning

Lochner

McEwen

Peiris

et al. 2016

ApJS

209

255

View full text Add to dashboard Cite

show abstract

“…Several methods were used to estimate photometric redshifts of galaxies or quasars like a K-nearest neighbors (e.g. Zhang et al (2013), Kügler et al (2015)), an artificial neural network (e.g. Firth et al (2003), Collister & Lahav (2004), Blake et al (2007), Oyaizu et al (2008), Yèche et al (2010), Zhang et al (2009)), both a K-nearest neighbors and a support vector machine (Han et al (2016)).…”

Section: Photometric Redshifts Of Quasarsmentioning

confidence: 99%

Deep learning approach for classifying, detecting and predicting photometric redshifts of quasars in the Sloan Digital Sky Survey stripe 82

Pasquet-Itam

Pasquet

2018

A&A

View full text Add to dashboard Cite

We apply a convolutional neural network (CNN) to classify and detect quasars in the Sloan Digital Sky Survey Stripe 82 and also to predict the photometric redshifts of quasars. The network takes the variability of objects into account by converting light curves into images. The width of the images, noted w, corresponds to the five magnitudes ugriz and the height of the images, noted h, represents the date of the observation. The CNN provides good results since its precision is 0.988 for a recall of 0.90, compared to a precision of 0.985 for the same recall with a random forest classifier. Moreover 175 new quasar candidates are found with the CNN considering a fixed recall of 0.97. The combination of probabilities given by the CNN and the random forest makes good performance even better with a precision of 0.99 for a recall of 0.90.For the redshift predictions, the CNN presents excellent results which are higher than those obtained with a feature extraction step and different classifiers (a K-nearest-neighbors, a support vector machine, a random forest and a gaussian process classifier). Indeed, the accuracy of the CNN within |∆z| < 0.1 can reach 78.09%, within |∆z| < 0.2 reaches 86.15%, within |∆z| < 0.3 reaches 91.2% and the value of rms is 0.359. The performance of the KNN decreases for the three |∆z| regions, since within the accuracy of |∆z| < 0.1, |∆z| < 0.2 and |∆z| < 0.3 is 73.72%, 82.46% and 90.09% respectively, and the value of rms amounts to 0.395. So the CNN successfully reduces the dispersion and the catastrophic redshifts of quasars. This new method is very promising for the future of big databases like the Large Synoptic Survey Telescope.

show abstract

“…Specifically, we use k-nearest neighbours (k-NN) regression, which is an intuitive method well-known in machine learning and to some extent also in astronomical communities (see, e.g., Li et al 2008;Polsterer et al 2013Polsterer et al , 2014Kügler et al 2015;Kremer et al 2015). Of the more prominent uses of k-NN in astronomy is the estimation of photo-z's in SDSS (Abazajian et al 2009).…”

Section: Increasing the Information From Photometric Measurementsmentioning

confidence: 99%

Sacrificing information for the greater good: how to select photometric bands for optimal accuracy

Stensbo-Smidt

Gieseke

Igel

et al. 2016

Mon. Not. R. Astron. Soc.

View full text Add to dashboard Cite

Large-scale surveys make huge amounts of photometric data available. Because of the sheer amount of objects, spectral data cannot be obtained for all of them. Therefore it is important to devise techniques for reliably estimating physical properties of objects from photometric information alone. These estimates are needed to automatically identify interesting objects worth a follow-up investigation as well as to produce the required data for a statistical analysis of the space covered by a survey. We argue that machine learning techniques are suitable to compute these estimates accurately and efficiently. This study promotes a feature selection algorithm, which selects the most informative magnitudes and colours for a given task of estimating physical quantities from photometric data alone. Using k nearest neighbours regression, a well-known non-parametric machine learning method, we show that using the found features significantly increases the accuracy of the estimations compared to using standard features and standard methods. We illustrate the usefulness of the approach by estimating specific star formation rates (sSFRs) and redshifts (photo-z's) using only the broadband photometry from the Sloan Digital Sky Survey (SDSS). For estimating sSFRs, we demonstrate that our method produces better estimates than traditional spectral energy distribution (SED) fitting. For estimating photo-z's, we show that our method produces more accurate photo-z's than the method employed by SDSS. The study highlights the general importance of performing proper model selection to improve the results of machine learning systems and how feature selection can provide insights into the predictive relevance of particular input features.

show abstract

Determining spectroscopic redshifts by usingknearest neighbor regression

Cited by 13 publications

References 21 publications

Photometric Supernova Classification With Machine Learning

Photometric Supernova Classification With Machine Learning

Deep learning approach for classifying, detecting and predicting photometric redshifts of quasars in the Sloan Digital Sky Survey stripe 82

Sacrificing information for the greater good: how to select photometric bands for optimal accuracy

Contact Info

Product

Resources

About