A sparse octree based parallel implementation of the lattice Boltzmann method for non-uniform meshes is presented in this paper. To couple grids of different resolutions, a second order accurate compact interpolation is employed and further extended into three dimensions. This compact interpolation requires only four source elements from the adjacent level for both two-and three dimensions.Thus, it reduces the computational and communication overhead in parallel executions. Moreover, the implementation of a weight based domain decomposition algorithm and level-wise elements arrangement are explained in details.The second order convergence of both velocity and strain rate are validated numerically in the Taylor-Green vortex test case. Additionally, the laminar flow around a cylinder at Re = 20, 100 and around a sphere at Re = 100 is investigated. Good agreement between simulated results and those from literature is observed, which provides further evidence for the accuracy of our method.