Adaptive Matrix Vector Product

Abstract

We consider the following streaming problem: given a hardwired m × n matrix A together with a poly(mn)-bit hardwired string of advice that may depend on A, for any x with coordinates x1, . . . xn presented in order, output the coordinates of A · x in order. Our focus is on using as little memory as possible while computing A · x; we do not count the size of the output tape on which the coordinates of A · x are written; for some matrices A such as the identity matrix, a constant number of words of space is achievable. Such an algorithm has to adapt its memory contents to the changing structure of A and exploit it on the fly. We give a nearly tight characterization, for any number of passes over the coordinates of x, of the space complexity of such a streaming algorithm. Our characterization is constructive, in that we provide an efficient algorithm matching our lower bound on the space complexity. The essential parameters, streaming rank and multi-pass streaming rank of A, might be of independent interest, and we show they can be computed efficiently. We give several applications of our results to computing Johnson-Lindenstrauss transforms. Finally, we note that we can characterize the optimal space complexity when the coordinates of A · x can be written in any order.