Querying a Matrix through Matrix-Vector Products

Abstract

We consider algorithms with access to an unknown matrix M ε F n×d via matrix-vector products , namely, the algorithm chooses vectors v 1 , ⃛ , v q , and observes Mv 1 , ⃛ , Mv q . Here the v i can be randomized as well as chosen adaptively as a function of Mv 1 , ⃛ , Mv i-1 . Motivated by applications of sketching in distributed computation, linear algebra, and streaming models, as well as connections to areas such as communication complexity and property testing, we initiate the study of the number q of queries needed to solve various fundamental problems. We study problems in three broad categories, including linear algebra, statistics problems, and graph problems. For example, we consider the number of queries required to approximate the rank, trace, maximum eigenvalue, and norms of a matrix M; to compute the AND/OR/Parity of each column or row of M, to decide whether there are identical columns or rows in M or whether M is symmetric, diagonal, or unitary; or to compute whether a graph defined by M is connected or triangle-free. We also show separations for algorithms that are allowed to obtain matrix-vector products only by querying vectors on the right, versus algorithms that can query vectors on both the left and the right. We also show separations depending on the underlying field the matrix-vector product occurs in. For graph problems, we show separations depending on the form of the matrix (bipartite adjacency versus signed edge-vertex incidence matrix) to represent the graph. Surprisingly, very few works discuss this fundamental model, and we believe a thorough investigation of problems in this model would be beneficial to a number of different application areas.

Cut query algorithms with star contraction

“…Not surprisingly, the aforementioned matrix-vector multiplication perspective also leads to efficient randomized algorithms for edge connectivity in the matrix-vector multiplication query model. While this model has been used previously in the study of sequential graph algorithms [OSV12] and (implicitly) in streaming algorithms for graph problems [AGM12], it began to be studied in and of itself relatively recently in the work of [SWYZ21], and has since seen several several follow-ups [CHL21,AL21]. More surprisingly, it turns out that the study of quantum algorithms with cut query access to a graph is also closely related to the matrix-vector multiplication model.…”

Low-Rank Approximation with $1/ε^{1/3}$ Matrix-Vector Products

“…Since this model captures a large family of iterative methods, it is natural to ask whether Krylov subspace based methods yield optimal algorithms, where the complexity measure of interest is the number of matrix-vector products. This model and related vector-matrix-vector query models were formalized for a number of problems in [SWYZ19,RWZ20], though the model is standard for measuring efficiency in scientific computing and numerical linear algebra, see, e.g., [BFG96]; in that literature, methods that use only matrix-vector products are called matrix-free. Subsequently, for the problem of estimating the top eigenvector, nearly tight bounds were obtained in [SAR18,BHSW20].…”

Average-Case Communication Complexity of Statistical Problems

“…A restriction of the sketching model, the u T Mv model, returns u M v for vectors u, v, where M is an unknown matrix [RWZ20]. This specializes the Mv model, which returns M v [SWYZ19]. These models all generalize edge-probes.…”