The aism (Approximate Inverse based on the Sherman-Morrison Formula) method is one of the existing effective methods for computing an approximate inverse. This algorithm was proposed by Bru et al. [SIAM J. Sci. Comput., 25, pp.701-715 (2003)]. Although it has been showed that the aism is generally a stable option for large linear systems of equations, its computation cost can be prohibitively high. Complications also arise when an attempt is made to parallelize the algorithm, since a sequential process is necessary. This article proposes a two level aism in which the coefficient matrix is rearranged to a block form, which is more suitable for parallel computation. This technique can also significantly speed-up computations on a single processor. We implemented this technique on an Origin 2400 system with an mpi to illustrate its efficiency through numerical experiments.