Neural Attention-driven Non-Maximum Suppression for Person Detection

Abstract

Non-maximum suppression (NMS) is a post-processing step in almost every visual object detector. NMS aims to prune the number of overlapping detected candidate regions-of-interest (ROIs) on an image, in order to assign a single and spatially accurate detection to each object. The default NMS algorithm (GreedyNMS) is fairly simple and suffers from severe drawbacks, due to its need for manual tuning. A typical case of failure with high application relevance is pedestrian/person detection in dense human crowds, where GreedyNMS doesn't provide accurate results. This paper proposes an efficient deep neural architecture for NMS in the person detection scenario, by capturing relations of neighbouring ROIs and aiming to ideally assign precisely one detection per person. The presented Seq2Seq-NMS architecture assumes a sequence-to-sequence formulation of the NMS problem, exploits the Multihead Scale-Dot Product Attention mechanism and jointly processes both geometric and visual properties of the input candidate ROIs. Thorough experimental evaluation on three public person detection datasets shows favourable results against competing methods, with acceptable inference runtime requirements and good behaviour for large numbers of raw candidate ROIs per image.

Auth-Persons: A Dataset for Detecting Humans in Crowds from Aerial Views

“…Evaluation of NMS Methods: YOLOv4-tiny was selected as the main person detector for the evaluation of the NMS methods. In this setup we compare the performance of the recently proposed Seq2Seq-NMS [20] and a wealth of other state-of-the-art NMS methods. The second competing method is a baseline Greedy-NMS approach running on CPU.…”

“…However, the visual data distribution shift between the training and the test samples may disproportionately affect in a negative manner those DNN-based NMS methods which do exploit appearance-based features in comparison to those who do not. Thus, Seq2Seq-NMS was evaluated in two variants: a) the vanilla Seq2Seq-NMS [20], and b) a trivial variant Seq2Seq-NMS geom which only exploits geometry-based features without considering visual appearance. Seq2Seq-NMS geom was implemented by simply feeding the DNN a zero vector for each ROI, as a dummy appearance-based feature.…”

“…Alternatively, Cluster-NMS was proposed in [19], i.e., a method where NMS is performed by implicitly clustering candidate detections, achieving very fast inference runtimes. Finally, a DNN-based NMS method called Seq2Seq-NMS [20] achieved top person detection results by assuming a sequence-to-sequence formulation of the NMS problem, exploiting the Multihead Scale-Dot Product Attention mechanism and jointly processing both geometric and visual properties of the input candidate detections.…”