A traditional Viterbi decoder is primarily optimized for additive white Gaussian noise (AWGN). With the AWGN channel, it offers good decoding performance. However, the underwater acoustic communication (UAC) channel is extremely complicated. In addition to white noise, there are a variety of artificial and natural impulse noise that occur suddenly. The traditional Viterbi decoder cannot obtain the optimum performance under this case. In order to solve this problem, this paper introduces a novel Viterbi decoder with the impulsive noise, which is considered to be subjected to Middleton Class A distribution in shallow ocean. Since Middleton Class A noise is very complicated, a simplified model is first introduced. Then, the error analysis of simplified model under various parameters is discussed in detail. The analysis shows that the simplified one just leads to slight error. Hereafter, a novel Veterbi decoder using the simplified model is discussed. Compared to a traditional decoder, a preprocessing is just required. The performance of soft decision-based decoder in the Middleton Class A noise channel (MAIN) and AWGN are further compared. Based on our simulations, the new decoder can significantly improve the performance in comparison with conventional one, which further validates our presented method.