We evaluated the clinical reliability and utility of temperature measurements using no-contact forehead infrared thermometers (NCFITs) by comparing their temperature measurements with those obtained using infrared tympanic thermometers (IRTTs) in children. In this observational, prospective, and cross-sectional study, we enrolled 255 children (aged 1 month to 18 years) from the pediatric surgery ward at a tertiary medical center in Korea. The mean age of the children was 9.05 ± 5.39 years, and 54.9% were boys. The incidence rate of fever, defined as an IRTT reading of ≥38.0 °C, was 15.7%. The ICC coefficient for the assessment of agreement between temperatures recorded by the NCFIT and IRTT was 0.87, and the κ-coefficient was 0.83. The bias and 95% limits of agreement were 0.15 °C (−0.43 to 0.73). For an accurate diagnosis of fever (≥38 °C), the false-negative rate was much lower, but the false-positive rate was higher, especially in 6-year-old children. Therefore, NCFITs can be used to screen children for fever. However, a secondary check is required using another thermometer when the child’s temperature is >38 °C. NCFITs are proposed for screening but not for measuring the temperature. For the latter, an accurate and reliable thermometer shall be used.