Need for a Careful Comparison Between Hypotheses: Case Study of Epicycles

Acosta, Griselda; Smith, Eric; Kreinovich, Vladik

doi:10.1007/978-3-030-46413-4_13

Cited by 2 publications

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…learners may indeed be better tools for explaining the intricacies of the world outside the window (if equipped with suitably large training corpora), but the successful development of theoretical physics depended on ignoring the world outside the window, so to speak, and looking into the night sky to study planetary orbits. Interestingly, the first models of planetary orbits in fact resemble the modern Machine Learning approach using function fits: the method of epicycles used in Ptolemy's astronomy to predict planetary orbits, are in fact equivalent to a Fourier series expansion [66]. Thus a key to theoretical advancements in physics has been to ignore the observed complexities of the natural world and to selectively focus on well-chosen details that provide the right hooks to unravel the seeming complexity.…”

Section: Increasing Rationalism Increasing Empiricismmentioning

confidence: 99%

Fitting Elephants

Mitra¹

2021

Preprint

View full text Add to dashboard Cite

Textbook wisdom advocates for smooth function fits and implies that interpolation of noisy data should lead to poor generalization. A related heuristic is that fitting parameters should be fewer than measurements (Occam's Razor). Surprisingly, contemporary machine learning (ML) approaches, cf. deep nets (DNNs), generalize well despite interpolating noisy data. This may be understood via Statistically Consistent Interpolation (SCI), i.e. data interpolation techniques that generalize optimally for big data. In this article we elucidate SCI using the weighted interpolating nearest neighbors (wiNN) algorithm, which adds singular weight functions to kNN (k-nearest neighbors). This shows that data interpolation can be a valid ML strategy for big data. SCI clarifies the relation between two ways of modeling natural phenomena: the rationalist approach (strong priors) of theoretical physics with few parameters and the empiricist (weak priors) approach of modern ML with more parameters than data. SCI shows that the purely empirical approach can successfully predict. However data interpolation does not provide theoretical insights, and the training data requirements may be prohibitive. Complex animal brains are between these extremes, with many parameters, but modest training data, and with prior structure encoded in species-specific mesoscale circuitry. Thus, modern ML provides a distinct epistemological approach different both from physical theories and animal brains.

show abstract

Section: Increasing Rationalism Increasing Empiricismmentioning

confidence: 99%