Large-scale area-based conservation measures affect millions of people globally. Understanding their social impacts is necessary to improve effectiveness and minimise negative consequences. However, quantifying the impacts of conservation measures that affect large geographic areas and diverse peoples is expensive and methodologically challenging, particularly because such evaluations should capture locally-defined conceptions of wellbeing while permitting policy-relevant comparisons. We measure the impact of Tanzania's Wildlife Management Areas (WMAs), a national community-based conservation and poverty reduction initiative. We used a novel, cost-effective impact evaluation method based on participatory wealth-ranking and Bayesian multilevel modelling. We find that from 2007-2015 the impacts of WMAs on wealth were small and variable, with no clear evidence of widespread poverty reduction. Accompanying qualitative data suggest that apparently positive effects in one WMA cannot be directly attributed to WMA activities. Our results suggest that current WMA policy needs to be revisited if it is to promote positive economic development. WMA name Region (Study Area) Year initiated Year gazetted Level of restrictions Generation of revenues