Nonlinear ultrasonic guided waves have attracted increasing attention in non-destructive evaluation (NDE) due to their advantages over linear ultrasonic guided waves, e.g. potential to be a baseline-free technique for damage detection and higher sensitivity to incipient damage in structures. This study presents an experimental investigation of using nonlinear ultrasonic guided waves to evaluate damage when reinforced concrete beams are subjected to high temperatures. In this study, the reinforced concrete beam specimens are heated up to high temperatures in a furnace and it shows that the high temperatures induce debonding between rebar and concrete. Ultrasonic guided wave inspection is carried out at different high temperatures. When the ultrasonic guided wave interacts with the debonding between rebar and concrete, the second harmonic, which is one of the nonlinear features of ultrasonic guided waves, is generated due to contact acoustic nonlinearity (CAN). In this study, a nonlinear parameter is defined to quantify the second harmonic generation due to different levels of debonding damage in the reinforced concrete beams at different temperatures. The relationship between the debonding damage caused by the high temperature and the characteristics of the nonlinear parameter is studied. The results show that the nonlinear parameter can provide an indication of the severity of the debonding damage, and can be used to monitor the growth of the debonding damage in the reinforced concrete beams at high temperatures.