Balancing biodiversity conservation and the socioeconomic well-being of ethnic minority communities presents a significant challenge in protected areas. Ecological compensation (EC) is a crucial policy instrument for improving the livelihood of affected residents, but its application in ethnic minority-dominant regions is complicated by cultural differences and fragile economic foundations. This study evaluates the impact of an innovative hybrid EC mechanism, integrating cash payment, employment support, and education incentive, on the livelihood resilience of ethnic minority groups and identifies the most cost-effective mechanism. Using field data from rural households in Potatso National Park, analyzed through hierarchical multiple regression and scenario simulations within the Sustainable Livelihood Framework, the study finds that: (i) Hybrid EC mechanisms significantly improve overall livelihoods, with households receiving employment support and education incentive showing 0.074 and 0.052 higher average livelihood score, respectively (on a 0-1 scale); (ii) Different EC types have varied effects across households; and (iii) Hybrid EC mechanisms increase social benefits by 6.97% to 37.65%, with only a 1.30% to 20.15% cost increase compared to baseline scenario. The findings highlight the need for diversified and optimized EC policies to maximize cost-effectiveness and improve livelihood resilience in protected areas.