Low-cost scalable discretization, prediction, and feature selection for complex systems

Abstract

Finding reliable discrete approximations of complex systems is a key prerequisite when applying many of the most popular modeling tools. Common discretization approaches (e.g., the very popular K-means clustering) are crucially limited in terms of quality, parallelizability, and cost. We introduce a low-cost improved quality scalable probabilistic approximation (SPA) algorithm, allowing for simultaneous data-driven optimal discretization, feature selection, and prediction. We prove its optimality, parallel efficiency, and a linear scalability of iteration cost. Cross-validated applications of SPA to a range of large realistic data classification and prediction problems reveal marked cost and performance improvements. For example, SPA allows the data-driven next-day predictions of resimulated surface temperatures for Europe with the mean prediction error of 0.75°C on a common PC (being around 40% better in terms of errors and five to six orders of magnitude cheaper than with common computational instruments used by the weather services).

Applications of matrix factorization methods to climate data

“…Similar generalizations can also be constructed for convex decompositions of the data. For instance, by virtue of the decomposability of the least squares cost function, it is possible to construct a joint convex discretization of instantaneous and lagged values of the variables of interest, which forms the basis of the scalable probabilistic approximation method (Gerber et al, 2020). In this approach, the decomposition may be parameterized in terms of a transition matrix relating the weights at different times, thus naturally incorporating a temporal constraint into the discretization.…”

“…7. The least restricted version (Lee and Seung, 1997;Gerber et al, 2020) requires only that the reconstruction lies in the convex hull of the basis, or, in other words, that the weights Z satisfy the constraints…”

“…2. In addition to providing a consistent framework for defining each method, it is straightforward to incorporate additional constraints or penalties, e.g., for the purposes of feature selection or to induce sparsity (Jolliffe et al, 2003;Lee et al, 2007;Mairal et al, 2009;Witten et al, 2009;Jenatton et al, 2010;Gerber et al, 2020). Solving the resulting (usually constrained) optimization problem amounts to learning a dictionary with which to represent the data, with different methods producing different dictionaries.…”

“…By carefully defining the optimization problem, the learned dictionary can be tuned to target particular features in the data. Below, we demonstrate this process by utilizing a recently introduced regularized convex coding (Gerber et al, 2020), which allows for feature selection to be performed by varying a regularization parameter. By tuning the imposed regularization to optimize the reconstruction or prediction error, the relative performance of selecting a basis lying on or outside the convex hull can be compared to one that preferentially extracts cluster means.…”

Utilizing Quantum Biological Techniques on a Quantum Processing Unit for Improved Protein Binding Site Determination

“…Most notably, the major difference that exists between modeling proteins and their binding sites on a quantum computer versus on any classical machine is the difference in complexity classes available to solve within. Looking at the most recently well-cited model for statistical modeling and classifications of proteins by Dr. Susanne Gerber at the University of Mainz [21], the highest degree of complexity achieved was a 3rd Degree polynomial time O(n) with bounds in Z. While this seems to be sufficient for a summation with 2 bounds, the complexity of completing this summation for an entire protein quickly becomes factorial, as each functional addition brings about another set of parameters to work through, as described here for just the kinetic energy of a protein, without taking into account the different material affects with each solvent.…”