Acoustic tomography (AT) is one of a few non-contact measurement techniques that can present information about the temperature distribution. Its successful application greatly depends on the performance of the reconstruction algorithm. In this paper, a temperature distribution reconstruction method based on compressed sensing (CS) is proposed. Firstly, a measurement matrix of an AT system in a CS framework is established. Secondly, a sparse basis is selected based on the mutual coherence between the measurement matrix and sparse basis. Thirdly, an improvement of the orthogonal matching pursuit (OMP) algorithm, called the IMOMP algorithm, is proposed for pursuing efficiency in recovering sparse signals. Reconstruction experiments of Gaussian sparse signals showed that IMOMP was better than OMP in both success ratio and running time, and the selection method of sparse basis was effective. Finally, a temperature distribution reconstruction algorithm based on compressed sensing, that is, the CS-IMOMP algorithm, is proposed. Simulation and experiment results show that, compared with the least square algorithm and the Simultaneous Iterative Reconstruction Technique algorithm, the CS-IMOMP algorithm has smaller reconstruction errors and provides more accurate information about the temperature distribution.