As the foremost step of spectral unmixing, endmember extraction has been one of the most challenging techniques in the spectral unmixing processing due to the mixing of pixels and the complexity of hyperspectral remote sensing images. The existing geometrial-based endmember extraction algorithms have achieved the ideal results, but most of these algorithms perform poorly when they do not meet the assumption of simplex structure. Recently, many intelligent optimization algorithms have been employed to solve the problem of endmember extraction. Although they achieved the better performance than the geometrial-based algorithms in different complex scenarios, they also suffer from the time-consuming problem. In order to alleviate the above problems, balance the two key indicators of accuracy and running time, an adaptive surrogate-assisted endmember extraction (ASAEE) framework based on intelligent optimization algorithms is proposed for hyperspectral remote sensing images in this paper. In the proposed framework, the surrogate-assisted model is established to reduce the expensive time cost of the intelligent algorithms by fitting the fully constrained evaluation value with the low-cost estimated value. In more detail, three commonly used intelligent algorithms, namely genetic algorithm, particle swarm optimization algorithm and differential evolution algorithm, are specifically designed into the ASAEE framework to verify the effectiveness and robustness. In addition, an adaptive weight surrogate-assisted model selection strategy is proposed, which can automatically adjust the weights of different surrogate models according to the characteristics of different intelligent algorithms. Experimental results on three data sets (including two simulated data sets and one real data set) show the effectiveness and the excellent performance of the proposed ASAEE framework.