Fast Monte Carlo Algorithms for Matrices I: Approximating Matrix Multiplication

Abstract

Motivated by applications in which the data may be formulated as a matrix, we consider algorithms for several common linear algebra problems. These algorithms make more efficient use of computational resources, such as the computation time, random access memory (RAM), and the number of passes over the data, than do previously known algorithms for these problems. In this paper, we devise two algorithms for the matrix multiplication problem. Suppose A and B (which are m × n and n × p, respectively) are the two input matrices. In our main algorithm, we perform c independent trials, where in each trial we randomly sample an element of {1, 2,. .. , n} with an appropriate probability distribution P on {1, 2,. .. , n}. We form an m × c matrix C consisting of the sampled columns of A, each scaled appropriately, and we form a c × n matrix R using the corresponding rows of B, again scaled appropriately. The choice of P and the column and row scaling are crucial features of the algorithm. When these are chosen judiciously, we show that CR is a good approximation to AB. More precisely, we show that AB − CR F = O(A F B F / √ c), where • F denotes the Frobenius norm, i.e., A 2 F = i,j A 2 ij. This algorithm can be implemented without storing the matrices A and B in RAM, provided it can make two passes over the matrices stored in external memory and use O(c(m + n + p)) additional RAM to construct C and R. We then present a second matrix multiplication algorithm which is similar in spirit to our main algorithm. In addition, we present a model (the pass-efficient model) in which the efficiency of these and other approximate matrix algorithms may be studied and which we argue is well suited to many applications involving massive data sets. In this model, the scarce computational resources are the number of passes over the data and the additional space and time required by the algorithm. The input matrices may be presented in any order of the entries (and not just row or column order), as is the case in many applications where, e.g., the data has been written in by multiple agents. In addition, the input matrices may be presented in a sparse representation, where only the nonzero entries are written.

Sampling Sub-problems of Heterogeneous Max-cut Problems and Approximation Algorithms

“…Note that establishing (23) in Theorem 3 (respectively, (11) in Theorem 2) uses ideas that are very similar to those used in [14] for approximating the product of two matrices. Once we are given (23) (respectively, (11)) then the proof of (21) (respectively, (9)) is immediate; we simply show that if the original LP has a solution then the sampled LP also has a solution sincePx is sufficiently close to Px.…”

“…Using the Select algorithm of [14] (a simple variant of reservoir sampling) we see that computing the probabilities (20) of Theorem 3 requires O(1) space [14]. On the other hand, computing…”

“…Such importance sampling has a long history [29]. In recent work, we have shown that by sampling columns and/or rows of a matrix according to a judiciouslychosen and data-dependent nonuniform probability distribution, we may obtain better (relative to uniform sampling) bounds for approximation algorithms for a variety of common matrix operations [14][15][16]; see also [10][11][12]. The power of using information to construct nonuniform sampling probabilities has also been demonstrated in recent work examining so-called oblivious versus so-called adaptive sampling [5,6].…”

“…The power of using information to construct nonuniform sampling probabilities has also been demonstrated in recent work examining so-called oblivious versus so-called adaptive sampling [5,6]. For instance, it was demonstrated that in certain cases approximation algorithms (for matrix problems such as those discussed in [14][15][16]) which use oblivious uniform sampling cannot achieve the error bounds that are achieved by adaptively constructing nonuniform sampling probabilities [5,6].…”

Approximating Function in the Application of the Approximate Calculation

“…In the engineering technology, the measurement of surveying and mapping, manufacturing, and other applications of science and technology, almost cannot leave the approximate calculation [1][2][3] , because accurate only exists in theory, can't get accurate value [4][5] in a specific application. Approximate calculation we all want to get as close as possible to the result of the accurate value, in order to achieve this goal, we all need to be used in the calculation of the approximate function [6][7] approximation precision value as much as possible, finally, the paper it can be seen, the most common basic approximation function is a rational function and polynomial function [8][9] .…”