Extracting road from high resolution remote sensing (HRRS) images is an economic and effective way to acquire road information, which has become an important research topic and has a wide range of applications. In this paper, we present a novel method for road extraction from HRRS images. Multi-kernel learning is first utilized to integrate the spectral, texture, and linear features of images to classify the images into road and non-road groups. A precise extraction method for road elements is then designed by building road shaped indexes to automatically filter out the interference of non-road noises. A series of morphological operations are also carried out to smooth and repair the structure and shape of the road element. Finally, based on the prior knowledge and topological features of the road, a set of penalty factors and a penalty function are constructed to connect road elements to form a complete road network. Experiments are carried out with different sensors, different resolutions, and different scenes to verify the theoretical analysis. Quantitative results prove that the proposed method can optimize the weights of different features, eliminate non-road noises, effectively group road elements, and greatly improve the accuracy of road recognition.