A Constructive, Type-Theoretic Approach to Regression via Global Optimisation

Abstract

We examine the connections between deterministic, complete, and general global optimisation of continuous functions and a general concept of regression from the perspective of constructive type theory via the concept of 'searchability'. We see how the property of convergence of global optimisation is a straightforward consequence of searchability. The abstract setting allows us to generalise searchability and continuity to higher-order functions, so that we can formulate novel convergence criteria for regression, derived from the convergence of global optimisation. All the theory and the motivating examples are fully formalised in the proof assistant AGDA. General:Algorithms may take into account information about the shape of the function. For example, the minimisation of functions with convex envelopes is intensively studied [27].In contrast, we will make minimal assumptions of this nature.Complete: An incomplete algorithm makes no guarantees regarding the quality of the solution it arrives at, focussing on efficiency via sophisticated heuristics, rather than correctness. The typical example of an incomplete algorithm is gradient descent, which will only find a local minimum of a function [23]. In contrast, we provide mathematical guarantees a solution is indeed optimal within some margin of error.Deterministic: A randomized algorithm can offer an asymptotic guarantee that the optimum is reached, with probability one, without actually knowing when it has been reached [26]. In contrast, we will give strong termination guarantees for the algorithm. Continuous:Many global optimisation algorithms deal with discrete problems, such as branchand-bound [14]. In contrast, we will focus on the minimisation of continuous functions.To summarise, in this paper we will concentrate on general, complete, continuous, deterministic global search, which finds one guaranteed optimal-within-epsilon global minimum of a continuous function [18]. In the sequel, by 'optimisation' this is what we mean, precisely.The first important results in the area relevant to our work appeared in the 1960s and 70s, the optimisation of rational functions using interval arithmetic by Moore and Young [16], which was 1