Abstract-Vectorless power grid verification is a powerful technique to validate the robustness of the on-chip power distribution network for all possible current waveforms. Formulated and solved as linear programming problems, vectorless power grid verification demands intensive computational power due to the large number of nodes in modern power grids. Previous work showed that the performance bottleneck of this powerful technique is within the sub-problem of power grid analysis, which essentially computes the inverse of the sparse but large power grid matrix. In this paper, we propose a hierarchical matrix inversion algorithm to compute the rows of the inverse efficiently by exploiting the structure of the power grid. The proposed algorithm is integrated with a previous dual algorithm addressing an orthogonal sub-problem for vectorless power grid verification. Results show that the proposed hierarchical algorithm accelerates the matrix inversion significantly, and thus makes the overall vectorless power grid verification efficient.