In practical applications, the intelligence of wheeled mobile robots is the trend of future development. Object detection for wheeled mobile robots requires not only the recognition of complex surroundings, but also the deployment of algorithms on resource-limited devices. However, the current state of basic vision technology is insufficient to meet demand. Based on this practical problem, in order to balance detection accuracy and detection efficiency, we propose an object detection algorithm based on a combination of improved YOLOv4 and improved GhostNet in this paper. Firstly, the backbone feature extraction network of original YOLOv4 is replaced with the trimmed GhostNet network. Secondly, enhanced feature extraction network in the YOLOv4, ordinary convolution is supplanted with a combination of depth-separable and ordinary convolution. Finally, the hyperparameter optimization was carried out. The experimental results show that the improved YOLOv4 network proposed in this paper has better object detection performance. Specifically, the precision, recall, F1, mAP (0.5) values, and mAP (0.75) values are 88.89%, 87.12%, 88.00%, 86.84%, and 50.91%, respectively. Although the mAP (0.5) value is only 2.23% less than the original YOLOv4, it is higher than YOLOv4_tiny, Eifficientdet-d0, YOLOv5n, and YOLOv5 compared to 29.34%, 28.99%, 20.36%, and 18.64%, respectively. In addition, it outperformed YOLOv4 in terms of mAP (0.75) value and precision, and its model size is only 42.5 MB, a reduction of 82.58% when compared to YOLOv4’s model size.