Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond

Abstract

Pedestrian detection is the cornerstone of many vision based applications, starting from object tracking to video surveillance and more recently, autonomous driving. With the rapid development of deep learning in object detection, pedestrian detection has achieved very good performance in traditional single-dataset training and evaluation setting. However, in this study on generalizable pedestrian detectors, we show that, current pedestrian detectors poorly handle even small domain shifts in cross-dataset evaluation. We attribute the limited generalization to two main factors, the method and the current sources of data. Regarding the method, we illustrate that biasness present in the design choices (e.g anchor settings) of current pedestrian detectors are the main contributing factor to the limited generalization. Most modern pedestrian detectors are tailored towards target dataset, where they do achieve high performance in traditional single training and testing pipeline, but suffer a degrade in performance when evaluated through cross-dataset evaluation. Consequently, a general object detector performs better in cross-dataset evaluation compared with state of the art pedestrian detectors, due to its generic design. As for the data, we show that the autonomous driving benchmarks are monotonous in nature, that is, they are not diverse in scenarios and dense in pedestrians. Therefore, benchmarks curated by crawling the web (which contain diverse and dense scenarios), are an efficient source of pre-training for providing a more robust representation. Accordingly, we propose a progressive fine-tuning strategy which improves generalization. Additionally, this work also investigate the recent Transformer Networks as backbones to test generalization. We demonstrate that as of now, CNNs outperform transformer networks in terms of generalization and absorbing large scale datasets for learning robust representation. In conclusion, this paper suggests a paradigm shift towards cross-dataset evaluation, for the next generation of pedestrian detectors. Code and models can be accessed at https://github.com/hasanirtiza/Pedestron.

Object detection using convolutional neural networks and transformer-based models: a review

“…Number of models can be made in conjunction with attention-based or transformerbased encoders or decoder modules with already existing object detectors. For example, in [62], PedesFormer is a swim transformer-based model that focuses on the advancement of research. Segmentation and domain adaptation are constructed using UNet network with swim transformer, as it can be applied spatial constraints to the pedestrian detection [13].…”