Since all kinds of noise exist in signals from real-world systems, it is very difficult to exactly estimate Lyapunov exponents from this time series. In this paper, a novel method for estimating the Lyapunov spectrum from a noisy chaotic time series is presented. We consider the higher-order mappings from neighbors into neighbors, rather than the mappings from small displacements into small displacements as usual. The influence of noise on the second-order mappings is researched, and an averaging method is proposed to cope with this noise. The mappings equations of the underlying deterministic system can be obtained from the noisy data via the method, and then the Lyapunov spectrum can be estimated. We demonstrate the performance of our algorithm for three familiar chaotic systems, Hénon map, the generalized Hénon map and Lorenz system. It is found that the proposed method provides a reasonable estimate of Lyapunov spectrum for these three systems when the noise level is less than 20%, 10% and 7%, respectively. Furthermore, our method is not sensitive to the distribution types of the noise, and the results of our method become more accurate with the increase of the length of time series.