Resource, environmental, and ecological issues have become major constraints to the development of many regions. The Yellow River Basin is an important barrier for maintaining ecological security in northern China, but it has been impacted by problems such as severe soil erosion and declining biodiversity. The rational construction of ecological security patterns is important to enhance ecosystem functions and maintain regional ecological security. In this study, a comprehensive ecological security assessment system was constructed by selecting ecosystem service importance, ecological sensitivity, and landscape connectivity to assess the ecological security of Xiliu Ditch, an ecologically fragile region of the Inner Mongolia section of the Yellow River Basin in China. The assessment results showed significant spatial heterogeneity, with medium- and low-security value areas dominating, while high-security value areas accounted for only 18.7% of the study area. Seventeen ecological sources were identified from the high-security areas, which were mainly composed of grassland, woodland, and water bodies, most of which are distributed in the southern part of the study area. Twenty ecological corridors were selected by the minimum cumulative resistance model and gravity model and classified into 15 construction corridors and 5 potential corridors. Forty-six ecological nodes were defined, including twenty strategic points, nine potential strategic points, and seventeen break points. On this basis, we constructed an ecological security pattern of “two belts, three cores, six zones, multiple corridors and multiple nodes” and proposed corresponding ecological governance measures. This study explores the ecological security pattern at the small watershed scale, which helps to realize the fine management of the Xiliu Ditch basin and, on this basis, can provide scientific support for the ecological protection and sustainable development of the Yellow River basin. In addition, the ecological security assessment system proposed in this study can provide new ideas for the construction of ecological security patterns in similar ecologically fragile areas around the globe.