The dynamic model about the anti-plane vibration of a contoured quartz plate with thickness changing continuously is established by ignoring the effect of small elastic constant c56. The governing equation is solved using the power series expansion technique, and the trapped thickness shear modes caused by bulge thickness are revealed. Theoretically, the proposed method is more general, which can be capable of handling various thickness profiles defined mathematically. After the convergence of the series is demonstrated and the correctness is numerically validated with the aid of finite element method results, systematic parametric studies are subsequently carried out to quantify the effects of the geometry parameter upon the trapped modes, including resonant frequency and mode shape. After that, the band structures of thickness shear waves propagation in a periodically contoured quartz plate, as well as the power transmission spectra, are obtained based on the power series expansion technique. It is revealed that broad stop bands below cut-off frequency exist owing to the trapped modes excited by the geometry inhomogeneity, which has little relationship with the structural periodicity, and its physical mechanism is different from the Bragg scattering effect. The outcome is widely applicable, and can be utilized to provide theoretical and practical guidance for the design and manufacturing of quartz resonators and wave filters.