Recently, spoof surface plasmon polaritons (SPPs) have been successfully applied to microwave signal propagation in ultra-thin planar circuits, due to their unique features of electromagnetic (EM) wave confinement in deep subwavelength scale and controllable dispersive properties. Both even and odd modes can be supported in symmetric spoof SPP structures. However, most device applications employ even mode propagation due to its easy stimulation. In this paper we investigate the dispersion characteristics and EM field patterns of both even and odd modes in a symmetric H-shaped spoof SPP structure. We concentrate on odd modes with an anti-symmetric electric field distribution, and their application to the propagation and filtering of differential signals. With odd modes, a balanced bandpass filter (BPF) can be constructed with a designable passband depending on the dispersion engineering of the odd modes in the spoof SPP structures. Pure odd mode propagation provides an ideal platform for differential signal transmission, with effective rejection of the common mode and an ultra-wide upper stopband in the designed filters. To validate the proposed design, two balanced BPFs operating at 2.3 GHz and 4 GHz are designed and implemented. There is good agreement between the simulated and measured results. We believe that the proposed method may encourage alternative applications of differential signal filtering with flexible frequency responses, and may play an important role in device development in wireless communication systems.