Satellite-based broad-scale (i.e., global and continental) human settlement data offer foundational information for diverse applications spanning climate hazard mitigation, sustainable development monitoring, spatial epidemiology, and demographic modeling. While many human settlement products report exceptional detection accuracies above 85%, there is a substantial blind spot in that product validation is typically centered on large urban areas rather than rural, small-scale settlements that are home to 3.4 billion people. In this study, we make use of a data-rich collection of 30 refugee settlements in Uganda to produce a targeted assessment of small-scale settlement detection by four broad-scale human settlement products: Global Human Settlements Built-Up Sentinel-2 (GHS-BUILT-S2), World Settlement Footprint (WSF), High Resolution Settlement Layer (HRSL), and Geo-Referenced Infrastructure and Demographic Data for Development (GRID3). We measured each product’s areal coverage within refugee settlements, assessed product detection accuracies in comparison to an independent dataset of 317,416 refugee settlement building footprints, and examined agreement between products. For refugee settlements established before 2016, the human settlement products had a low median F1-Score (F1) of 0.24, a high median false alarm rate of 0.59, and tended to only agree at locations of highest building density. Individually, WSF entirely overlooked 8 of the 30 study refugee settlements (median F1=0.17); GHS-BUILT-S2 underestimated the building footprint area by a median 50% (F1=0.15); GRID3 overestimated the building footprint area by a median 280% (F1=0.38); and HRSL underestimated the median area by 7% (F1=0.34). All products suffer from low detection accuracy and high false alarm rates, but GRID3 and HRSL, based on 0.5 meter resolution imagery, offer better detection accuracy than GHS-BUILT S2 and WSF, which are based on 10-30 meter resolution imagery. These results show that human settlement products have far to go in providing an accurate depiction of small-scale refugee settlements and would benefit from incorporating refugee settlements in training and validation of broad-scale human settlement detection.