Multidimensional indexing tools for the virtual observatory

Csabai, István; Dobos, László; Trencseni, Marton; Herczegh, G.; Józsa, Péter; Purger, Norbert; Budavàri, Tamás; Szalay, Alexander S.

doi:10.1002/asna.200710817

Cited by 77 publications

(95 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This assumption is then used in an algorithm for estimating galaxies with unknown redshifts. Examples of machine learning tools used for this purpose include artificial neural networks (Collister et al 2007;Reis et al 2012;Brescia et al 2014), local polynomial fits (Csabai et al 2007), random forests (Carliles et al 2010), and boosted decision trees (Gerdes et al 2010).…”

Section: Photometric Redshift Estimationmentioning

confidence: 99%

“…Following Csabai et al (2007) and earlier SDSS releases, we adopted a local (or piecewise) linear model to describe how the redshifts of galaxies depend on broad-band colours and magnitudes. The locality allows the model to follow the complex relationship between these properties, while using a polynomial of just the first order means that a relatively small number of galaxies is enough to fit the parameters.…”

Section: Local Linear Regressionmentioning

confidence: 99%

See 1 more Smart Citation

Photometric redshifts for the SDSS Data Release 12

Beck

Dobos

Budavàri

et al. 2016

Mon. Not. R. Astron. Soc.

189

248

View full text Add to dashboard Cite

We present the methodology and data behind the photometric redshift database of the Sloan Digital Sky Survey Data Release 12 (SDSS DR12). We adopt a hybrid technique, empirically estimating the redshift via local regression on a spectroscopic training set, then fitting a spectrum template to obtain K-corrections and absolute magnitudes. The SDSS spectroscopic catalog was augmented with data from other, publicly available spectroscopic surveys to mitigate target selection effects. The training set is comprised of 1, 976, 978 galaxies, and extends up to redshift z ≈ 0.8, with a useful coverage of up to z ≈ 0.6. We provide photometric redshifts and realistic error estimates for the 208, 474, 076 galaxies of the SDSS primary photometric catalog. We achieve an average bias of ∆z norm = 5.84 × 10 −5 , a standard deviation of σ (∆z norm ) = 0.0205, and a 3σ outlier rate of P o = 4.11% when cross-validating on our training set. The published redshift error estimates and photometric error classes enable the selection of galaxies with high quality photometric redshifts. We also provide a supplementary error map that allows additional, sophisticated filtering of the data.

show abstract

Section: Photometric Redshift Estimationmentioning

confidence: 99%

Section: Local Linear Regressionmentioning

confidence: 99%

Photometric redshifts for the SDSS Data Release 12

Beck

Dobos

Budavàri

et al. 2016

Mon. Not. R. Astron. Soc.

189

248

View full text Add to dashboard Cite

show abstract

“…the galaxies for which we want to estimate redshift) and then estimates the redshift by fitting a local low order polynomial to these points. An improved version of this code is using a k-d tree index for fast nearest neighbour search (Csabai et al 2007). It was used to calculate photometric redshifts for the SDSS Data Release 7 (Abazajian et al 2009).…”

Section: Purger (Nearest-neighbour Fit) (Pn-e)mentioning

confidence: 99%

PHAT: PHoto-zAccuracy Testing

Hildebrandt

Arnouts

Capak

et al. 2010

A&A

Self Cite

255

306

View full text Add to dashboard Cite

Context. Photometric redshifts (photo-z's) have become an essential tool in extragalactic astronomy. Many current and upcoming observing programmes require great accuracy of photo-z's to reach their scientific goals. Aims. Here we introduce PHAT, the PHoto-z Accuracy Testing programme, an international initiative to test and compare different methods of photo-z estimation. Methods. Two different test environments are set up, one (PHAT0) based on simulations to test the basic functionality of the different photo-z codes, and another one (PHAT1) based on data from the GOODS survey including 18-band photometry and ∼2000 spectroscopic redshifts.Results. The accuracy of the different methods is expressed and ranked by the global photo-z bias, scatter, and outlier rates. While most methods agree very well on PHAT0 there are differences in the handling of the Lyman-α forest for higher redshifts. Furthermore, different methods produce photo-z scatters that can differ by up to a factor of two even in this idealised case. A larger spread in accuracy is found for PHAT1. Few methods benefit from the addition of mid-IR photometry. The accuracy of the other methods is unaffected or suffers when IRAC data are included. Remaining biases and systematic effects can be explained by shortcomings in the different template sets (especially in the mid-IR) and the use of priors on the one hand and an insufficient training set on the other hand. Some strategies to overcome these problems are identified by comparing the methods in detail. Scatters of 4-8% in Δz/(1 + z) were obtained, consistent with other studies. However, somewhat larger outlier rates (>7.5% with Δz/(1 + z) > 0.15; >4.5% after cleaning) are found for all codes that can only partly be explained by AGN or issues in the photometry or the spec-z catalogue. Some outliers were probably missed in comparisons of photo-z's to other, less complete spectroscopic surveys in the past. There is a general trend that empirical codes produce smaller biases than template-based codes. Conclusions. The systematic, quantitative comparison of different photo-z codes presented here is a snapshot of the current state-ofthe-art of photo-z estimation and sets a standard for the assessment of photo-z accuracy in the future. The rather large outlier rates reported here for PHAT1 on real data should be investigated further since they are most probably also present (and possibly hidden) in many other studies. The test data sets are publicly available and can be used to compare new, upcoming methods to established ones and help in guiding future photo-z method development.

show abstract

“…The SDSS photometric redshifts are generated using a hybrid technique of the template method (Budavári et al 2000) and a machine learning component using k-nearest neighbours (Csabai et al 2007) technique as described in Abazajian et al (2009). We hereafter refer to this combined method as 'template-ml'.…”

Section: Sdss Dr10 Photometric Redshiftsmentioning

confidence: 99%

Feature importance for machine learning redshifts applied to SDSS galaxies

Hoyle

Rau

Zitlau

et al. 2015

Monthly Notices of the Royal Astronomical Society

View full text Add to dashboard Cite

We present an analysis of importance feature selection applied to photometric redshift estimation using the machine learning architecture Decision Trees with the ensemble learning routine Adaboost (hereafter RDF). We select a list of 85 easily measured (or derived) photometric quantities (or 'features') and spectroscopic redshifts for almost two million galaxies from the Sloan Digital Sky Survey Data Release 10. After identifying which features have the most predictive power, we use standard artificial Neural Networks (aNN) to show that the addition of these features, in combination with the standard magnitudes and colours, improves the machine learning redshift estimate by 18% and decreases the catastrophic outlier rate by 32%. We further compare the redshift estimate using RDF with those from two different aNNs, and with photometric redshifts available from the SDSS. We find that the RDF requires orders of magnitude less computation time than the aNNs to obtain a machine learning redshift while reducing both the catastrophic outlier rate by up to 43%, and the redshift error by up to 25%. When compared to the SDSS photometric redshifts, the RDF machine learning redshifts both decreases the standard deviation of residuals scaled by 1/(1+z) by 36% from 0.066 to 0.041, and decreases the fraction of catastrophic outliers by 57% from 2.32% to 0.99%.

show abstract

Multidimensional indexing tools for the virtual observatory

Cited by 77 publications

References 12 publications

Photometric redshifts for the SDSS Data Release 12

Photometric redshifts for the SDSS Data Release 12

PHAT: PHoto-zAccuracy Testing

Feature importance for machine learning redshifts applied to SDSS galaxies

Contact Info

Product

Resources

About