Mapping the density contrast through the 3D gravity inversion can help detect the goals under the subsurface. However, it is a challenge to accurately and efficiently solve the 3D gravity inversion. The Krylov subspace method is widely used to solve large linear problems because of its small storage capacity and high computational efficiency. In this study, two classical algorithms of Krylov subspace method, i.e., the Generalized Minimum Residual method and the Conjugate Gradient method, are applied to 3D gravity inversion. Based on the results of the deep mineral model and the shallow L-shaped tunnel model tests, the Generalized Minimum Residual method was found to provide density contrast results similar to the Conjugate Gradient method. The distribution position of the density contrast obtained by inversion corresponds well to the position of the deep mineral resources model and the L-shaped tunnel model. The 3D underground distribution of Fe content is obtained by performing inversion of the measured gravity data of the Olympic Dam in Australia, and the obtained results correspond well with the distribution position of Fe content in the collected geological profile. Under the same conditions, the inversion accuracy of the Generalized Minimum Residual method is similar to that of the Conjugate Gradient method. However, the Generalized Minimum Residual method has faster convergence speed than the Conjugate Gradient method, and the inversion efficiency is increased by about 90%, which greatly reduces the inversion time and improves the inversion efficiency.