Accurate livestock counts are essential for effective pastureland management. High spatial resolution remote sensing, coupled with deep learning, has shown promising results in livestock detection. However, challenges persist, particularly when the targets are small and in a heterogeneous environment, such as those in African rangelands. This study evaluated nine state-of-the-art object detection models, four variants each from YOLOv5 and YOLOv8, and Faster R-CNN, for detecting cattle in 10 cm resolution aerial RGB imagery in Kenya. The experiment involved 1039 images with 9641 labels for training from sites with varying land cover characteristics. The trained models were evaluated on 277 images and 2642 labels in the test dataset, and their performance was compared using Precision, Recall, and Average Precision (AP0.5–0.95). The results indicated that reduced spatial resolution, dense shrub cover, and shadows diminish the model’s ability to distinguish cattle from the background. The YOLOv8m architecture achieved the best AP0.5–0.95 accuracy of 39.6% with Precision and Recall of 91.0% and 83.4%, respectively. Despite its superior performance, YOLOv8m had the highest counting error of −8%. By contrast, YOLOv5m with AP0.5–0.95 of 39.3% attained the most accurate cattle count with RMSE of 1.3 and R2 of 0.98 for variable cattle herd densities. These results highlight that a model with high AP0.5–0.95 detection accuracy may struggle with counting cattle accurately. Nevertheless, these findings suggest the potential to upscale aerial-imagery-trained object detection models to satellite imagery for conducting cattle censuses over large areas. In addition, accurate cattle counts will support sustainable pastureland management by ensuring stock numbers do not exceed the forage available for grazing, thereby mitigating overgrazing.