On the Bias-Variance Tradeoff: Textbooks Need an Update

Neal, Brady

doi:10.48550/arxiv.1912.08286

Cited by 4 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This results in many predictions with high bias and low variance. For example, the bias-variance tradeoff for the KNN algorithm varies depending on the number of nearest neighbours used, where using low values for the number of nearest neighbours can induce overfitting whilst using high values for the number of nearest neighbours can induce underfitting (Valencia-Zapata et al 2017;Neal 2019). For the SRKNN algorithm, we examine a wide range of number of nearest neighbours from 1 to 25 but this range could be extended with increased computation in future work to explore a larger number of nearest neighbours.…”

Section: Practicality Of the Machine Learning Techniques Usedmentioning

confidence: 99%

Z-Sequence: photometric redshift predictions for galaxy clusters with sequential random k-nearest neighbours

Chan

Stott

2021

Monthly Notices of the Royal Astronomical Society

View full text Add to dashboard Cite

We introduce Z-Sequence, a novel empirical model that utilises photometric measurements of observed galaxies within a specified search radius to estimate the photometric redshift of galaxy clusters. Z-Sequence itself is composed of a machine learning ensemble based on the k-nearest neighbours algorithm. We implement an automated feature selection strategy that iteratively determines appropriate combinations of filters and colours to minimize photometric redshift prediction error. We intend for Z-Sequence to be a standalone technique but it can be combined with cluster finders that do not intrinsically predict redshift, such as our own DEEP-CEE. In this proof-of-concept study we train, fine-tune and test Z-Sequence on publicly available cluster catalogues derived from the Sloan Digital Sky Survey. We determine the photometric redshift prediction error of Z-Sequence via the median value of |Δz|/(1 + z) (across a photometric redshift range of 0.05 ≤ z ≤ 0.6) to be ∼0.01 when applying a small search radius. The photometric redshift prediction error for test samples increases by 30-50 per cent when the search radius is enlarged, likely due to line-of-sight interloping galaxies. Eventually, we aim to apply Z-Sequence to upcoming imaging surveys such as the Legacy Survey of Space and Time to provide photometric redshift estimates for large samples of as yet undiscovered and distant clusters.

show abstract

Section: Practicality Of the Machine Learning Techniques Usedmentioning

confidence: 99%

Z-Sequence: photometric redshift predictions for galaxy clusters with sequential random k-nearest neighbours

Chan

Stott

2021

Monthly Notices of the Royal Astronomical Society

View full text Add to dashboard Cite

show abstract

“…A common way to interpret the VC dimension bound is in terms of bias-variance tradeoff (Neal et al, 2018;Neal, 2019), which is a simple heuristic that is widely used in machine learning. Bias-variance tradeoff captures the intuition that there is a tradeoff between a model being too simple, when it cannot properly represent the data (large bias), and a model being too complex (large capacity) when it will tend to overfit, leading to large variance on unseen data.…”

Section: Comparing Existing Bounds Against Desideratamentioning

confidence: 99%

Generalization bounds for deep learning

Valle-Pérez,

Louis

2020

Preprint

View full text Add to dashboard Cite

Generalization in deep learning has been the topic of much recent theoretical and empirical research. Here we introduce desiderata for techniques that predict generalization errors for deep learning models in supervised learning. Such predictions should 1) scale correctly with data complexity; 2) scale correctly with training set size; 3) capture differences between architectures; 4) capture differences between optimization algorithms; 5) be quantitatively not too far from the true error (in particular, be non-vacuous); 6) be efficiently computable; and 7) be rigorous. We focus on generalization error upper bounds, and introduce a categorisation of bounds depending on assumptions on the algorithm and data. We review a wide range of existing approaches, from classical VC dimension to recent PAC-Bayesian bounds, commenting on how well they perform against the desiderata. We next use a function-based picture to derive a marginal-likelihood PAC-Bayesian bound. This bound is, by one definition, optimal up to a multiplicative constant in the asymptotic limit of large training sets, as long as the learning curve follows a power law, which is typically found in practice for deep learning problems. Extensive empirical analysis demonstrates that our marginal-likelihood PAC-Bayes bound fulfills desiderata 1-3 and 5. The results for 6 and 7 are promising, but not yet fully conclusive, while only desideratum 4 is currently beyond the scope of our bound. Finally, we comment on why this function-based bound performs significantly better than current parameter-based PAC-Bayes bounds.

show abstract

Starting the Conversation Around the Ethical Use of Artificial Intelligence in Applied Behavior Analysis

Jennings,

Cox

2023

Behav Analysis Practice

View full text Add to dashboard Cite

Artificial intelligence (AI) is increasingly a part of our everyday lives. Though much AI work in healthcare has been outside of applied behavior analysis (ABA), researchers within ABA have begun to demonstrate many different ways that AI might improve the delivery of ABA services.Though AI offers many exciting advances, absent from the literature thus far is conversation around ethical considerations when developing, building, and deploying AI technologies.Further, though AI is already in the process of coming to ABA, it is unknown the extent to which behavior analytic practitioners are familiar (and comfortable) with the use of AI in ABA. The purpose of this paper is twofold. First, to describe how existing ethical publications (e.g., BACB Code of Ethics) do and do not speak to the unique ethical concerns with deploying AI in ABA.Second, to raise questions for consideration that might inform future ethical guidelines when developing and using AI in ABA service delivery. In total, we hope this paper sparks proactive dialog around the ethical use of AI in ABA before the field is required to have a reactionary conversation.

show abstract

On the Bias-Variance Tradeoff: Textbooks Need an Update

Cited by 4 publications

References 0 publications

Z-Sequence: photometric redshift predictions for galaxy clusters with sequential random k-nearest neighbours

Z-Sequence: photometric redshift predictions for galaxy clusters with sequential random k-nearest neighbours

Generalization bounds for deep learning

Starting the Conversation Around the Ethical Use of Artificial Intelligence in Applied Behavior Analysis

Contact Info

Product

Resources

About