Airport runway recognition technology would play an important role in developing intelligent weapon systems in the future. In this letter, a method of automatically finding runways in forward looking infrared (FLIR) images is proposed based on the knowledge of vision. First, the line segments in the images are extracted by a fast line segment detector (LSD) and an improved line segment linking method. Then, the regions of interest (ROI) of runways are detected using some constraint rules based on the direction, gradient, and width of line segment pairs. Afterward, an ROI length backtracking technique based on texture distribution is presented to retrieve the complete ROI. Finally, using runway regional self-similarity and contextual information, several decision criteria are formulated to accurately recognize the runway. Experimental results on the FLIR images with different imaging ranges show that the proposed algorithm is robust and has a good real-time performance.Index Terms-Airport runway recognition, forward looking infrared (FLIR) images, linear feature extraction, region-of-interest (ROI) length backtracking, texture similarity.