Antenna feed, as a crucial component of High Power Microwave (HPM) systems, plays a vital role in facilitating the transmission of microwave from vacuum environment to atmospheric space. However, the feed HPM breakdown has become as a frequent fault phenomenon evidently affecting the HPM system stability and output. In this paper, the abnormal waveform during a feed of a HPM simulator test has been studied, it was found that there was severe electric field distortion around the dielectric window fixing screw in the simulation of the feed, much lager than the breakdown threshold. To address this issue, the nylon material screws were employed as replacements for metal screws. In addition, the novel flute in the surface of the dielectric window is designed. The single-shot and repeated tests demonstrated that the improvement were effective, as the fault phenomenon was completely eradicated.