Radial basis function interpolation is often employed in mesh deformation algorithms for unstructured meshes, for example in fluid-structure interaction or design optimization problems. This is known to be a robust methodology that results in high quality deformed meshes. The applicability of this method to large problems is currently hampered by its prohibitive computational cost, however, which is caused by the need to solve a dense system of equations. The computation time grows as O(N 3 b) if a direct solver is employed for the solution (where N b denotes the number of boundary nodes in the mesh), while alternative iterative solvers often suffer from an unfavorable convergence behavior. In this paper, we present the inverse fast multipole method as a novel fast approximate direct solver with a computational cost scaling as O(N b). The linear complexity is achieved by transforming the dense system into an extended sparse system, along with the compression of certain matrix blocks into low-rank factorizations. The solver is inexact, although the error can be controlled and made as small as needed; a low accuracy solver can hence be used as an efficient preconditioner in an iterative scheme. Numerical benchmarks are presented, demonstrating that the proposed approach enhances the computational efficiency of mesh deformation algorithms based on radial basis function interpolation significantly, without jeopardizing their robustness and quality.