In this paper, we propose an effective method for remote sensing image registration. Point features are robust to remote sensing images with low quality, small overlapping area, and local deformation. Therefore, we extract point features from remote sensing images and convert the problem of remote sensing image registration into the problem of feature point matching. A correspondence set constructed solely on the similar of features often contains many false correspondences or outliers, so our key idea is to remove the mismatches in the initial correspondence set and obtain a stable correspondence through a two-step strategy. First, we use two constraints to construct the optimization model which can solve in linear time. The first constraint is that the topology of the points and their neighbors can be maintained after the spatial transformation. Another constraint is that the feature distance of the correct matches are similar to the neighbors. Then, we design a strategy to increase the number of inliers and raise the precision by a global constraint calculated from the solution in the previous step. Experiments on a variety of remote sensing image datasets demonstrate that our method is more robust and accurate than state-of-the-art methods.