Abstract-The mode transition in a coated long-period fiber grating (LPFG) has been studied based on coupled-mode theory. The response of cladding mode effective index with increasing overlay thickness is analyzed. The lower order modes are transferred to the overlay, and the higher order modes replace the lower order modes order by order and continue to propagate in the cladding. Further, the transmission spectrum of the coated LPFG is investigated while the overlay thickness is located in mode transition and the surrounding region, the amplitude of attenuation peak of the cladding mode which will enter the overlay decreases rapidly and the peak position of adjacent higher cladding mode shift towards the former lower mode. To design high sensitivity LPFG sensor, the optimal overlay thickness is selected to ensure that the coated LPFG is located in the mode fast transition region. Finally, the response characteristics of film refractive index of coated LPFG were investigated for two typical cladding modes, and two kinds of the sensor sensitivity are defined and calculated. The results indicate that the resolution of film refractive index in mode transition can be available to 10 -6 and 10 -7 respectively, if the sensor is detected by transmittance change and wavelength shift.