Based on the mathematical analysis of band transmission signal coherent detection theory and mud pressure DPSK signal modulation process, by establishing the coherent detection mathematical model of mud pressure DPSK signal, this paper studies the practicability of broadband mud pressure DPSK signal coherent detection. By analyzing the noise impact on input signal parameter of demodulation system, based on probabilistic statistics theory and coherent demodulation mathematical model, the bit error rate and antinoise ability of mud pressure DPSK signal is studied. Theoretical analysis and numerical simulation indicate that, by adding arc cosine and derivation segment to basic demodulation process, the coherent demodulation effect of mud pressure DPSK signal is satisfied, but the demodulation process is more complex than conventional communication system. Theoretical analysis shows that mud pressure DPSK signal has the same theatrical bit error rate as conventional communication binary phase shift keying (2PSK) signal, but due to the difference of signal demodulation means, the reconstruction of rotary valve control pulse will be affected seriously by the derivate of vertical component of the noise within frequency band, causing that the antinoise ability of mud pressure DPSK signal demodulation system is far lower than 2PSK signal system, and numerical calculation shows that the actual bit error rate is far higher than latter; so under the condition of large carrier frequency, to obtain relative small actual bit error rate and relative larger information transmission rate, the mud pressure DPSK signal input signal-to-noise ratio should be raised as far as possible.