Efficient Adaptive Online Learning via Frequent Directions

Wan, Yuanyu; Zhang, Lijun

doi:10.1109/tpami.2021.3096880

Cited by 2 publications

(1 citation statement)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, it requires unacceptable time and space to directly compute XY T in the main memory. To reduce both time and space complexities, approximate matrix multiplication (AM-M) with limited space, which can efficiently compute a good approximation of the matrix product, has been a substitute and received ever-increasing attention (Ye, Luo, and Zhang 2016;Wan, Wei, and Zhang 2018;Kuzborskij, Cella, and Cesa-Bianchi 2019).…”

Section: Introductionmentioning

confidence: 99%

Approximate Multiplication of Sparse Matrices with Limited Space

Wan

Zhang

2021

AAAI

Self Cite

View full text Add to dashboard Cite

Approximate matrix multiplication with limited space has received ever-increasing attention due to the emergence of large-scale applications. Recently, based on a popular matrix sketching algorithm---frequent directions, previous work has introduced co-occuring directions (COD) to reduce the approximation error for this problem. Although it enjoys the space complexity of O((m_x+m_y)l) for two input matrices X∈ℝ^{m_x ╳ n} and Y∈ℝ^{m_y ╳ n} where l is the sketch size, its time complexity is O(n(m_x+m_y+l)l), which is still very high for large input matrices. In this paper, we propose to reduce the time complexity by exploiting the sparsity of the input matrices. The key idea is to employ an approximate singular value decomposition (SVD) method which can utilize the sparsity, to reduce the number of QR decompositions required by COD. In this way, we develop sparse co-occuring directions, which reduces the time complexity to Õ((nnz(X)+nnz(Y))l+nl^2) in expectation while keeps the same space complexity as O((m_x+m_y)l), where nnz(X) denotes the number of non-zero entries in X and the Õ notation hides constant factors as well as polylogarithmic factors. Theoretical analysis reveals that the approximation error of our algorithm is almost the same as that of COD. Furthermore, we empirically verify the efficiency and effectiveness of our algorithm.

show abstract

Section: Introductionmentioning

confidence: 99%