Benchmarking and scalability of machine-learning methods for photometric redshift estimation

Abstract

Obtaining accurate photometric redshift (photo-z) estimations is an important aspect of cosmology, remaining a prerequisite of many analyses. In creating novel methods to produce photo-z estimations, there has been a shift towards using machine learning techniques. However, there has not been as much of a focus on how well different machine learning methods scale or perform with the ever-increasing amounts of data being produced. Here, we introduce a benchmark designed to analyse the performance and scalability of different supervised machine learning methods for photo-z estimation. Making use of the Sloan Digital Sky Survey (SDSS - DR12) dataset, we analysed a variety of the most used machine learning algorithms. By scaling the number of galaxies used to train and test the algorithms up to one million, we obtained several metrics demonstrating the algorithms’ performance and scalability for this task. Furthermore, by introducing a new optimisation method, time-considered optimisation, we were able to demonstrate how a small concession of error can allow for a great improvement in efficiency. From the algorithms tested we found that the Random Forest performed best with a mean squared error, MSE = 0.0042; however, as other algorithms such as Boosted Decision Trees and k-Nearest Neighbours performed very similarly, we used our benchmarks to demonstrate how different algorithms could be superior in different scenarios. We believe benchmarks like this will become essential with upcoming surveys, such as the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST), which will capture billions of galaxies requiring photometric redshifts.

Extracting photometric redshift from galaxy flux and image data using neural networks in the CSST survey

“…Inception blocks proposed by Szegedy et al (2014) can extract information in different scales parallelly and effectively combine them. Pasquet et al (2019) and Henghes et al (2021) and our previous work build their networks based on inception block to predict photometric redshift from images and achieve quite accurate results. Therefore, we construct Bayesian inception blocks with flipout layers.…”

Explaining deep learning of galaxy morphology with saliency mapping

“…The larger size of the Galaxy Zoo dataset allowed more general learning of galactic features and helped increase the predictiveness of the XAI based methods. A smaller dataset would reduce the performance of the initial model (Henghes et al 2021) and thus would reduce the ability of XAI methods to extract galactic features.…”

Investigating Deep Learning Methods for Obtaining Photometric Redshift Estimations from Images

“…A convolutional neural network (CNN) and an inception module CNN which both used the image data as the input. A random forest (RF) and extremely randomised trees (ERT) that had previously been found to be the best performing traditional methods (Henghes et al 2021) and which only used magnitude features. And two experimental mixed-input models, which combined a CNN or inception module CNN with a multi-layer perception to use both the image data and magnitude features as inputs.…”

“…Benchmarking is the process of running a set of standardised tests to determine the relative performance of an object, in this case iteratively running the training and testing of different machine learning algorithms. Here, benchmarking was performed in a similar vein to Henghes et al (2021). We recorded the time taken throughout the machine learning process and varied the size of the training dataset to be able to compare the efficiency of the various models.…”