With the development of medium-and high-resolution satellites, successfully segmenting differently sized geo-objects remains a challenging issue in the framework of geographic object-based image analysis (GEOBIA). The hybrid image segmentation method is a good alternative to produce good segmentation that best matched the different sizes of geo-objects. However, the existing methods almost use a certain scale or other segmentation parameters (SPs) to control the sizes and shapes of segments. This will lead to two issues: (1) one single scale is impossible to segment every geo-object well due to the land cover complexity within remote-sensing imageries; (2) over-and under-segmented regions still occur in the segmentation results, whatever using any advanced segmentation methods. To solve the above problems, this paper developed a hybrid image segmentation method with local scale-guided hierarchical region merging and further over-and under-segmentation processing. First, the primitive segmentation was produced and then stratified into layers with different land covers. Then, the local scale was calculated for a more objective merging process in the separating layers. Third, the over-and under-segmentation at separating layers was recognized and re-processed for achieving a fine segmentation. To validate the proposed method, it was applied to three test images of gaofen-1 satellite with different land cover types, and ten competing methods were compared. The visual and quantitative results indicated the advantage of our method in segmenting out different sizes of geo-objects, which can effectively reduce the over-and under-segmentation error.INDEX TERMS geographic object-based image analysis (GEOBIA), hybrid image segmentation, local scale, hierarchical region merging, over-and under-segmentation recognition and re-process.