SpotNet: Self-Attention Multi-Task Network for Object Detection

Perreault, Hughes; Bilodeau, Guillaume-Alexandre; Saunier, Nicolas; Héritier, Maguelonne

doi:10.1109/crv50864.2020.00038

Cited by 45 publications

(24 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, there is no need to implement an NMS for the final inference, which repeatedly computes the IOU between the estimated and observed bounding boxes. As an extension of a CenterNet, some researchers developed a SpotNet model by adding a head for semantic segmentation to its architecture [22]. When training the model, the head for semantic segmentation is fed with a silhouette derived by a background subtraction method using consecutive video shoots.…”

Section: Related Workmentioning

confidence: 99%

“…The performance of YOLO models ranges between 0.6 and 0.8 when measured by the mAP score. CenterNet and its extension SpotNet have also been adopted in studies of vehicle detection using the UA-DTRAC dataset [3], [22], [31]- [32]. These two key-point-based detection models have recorded mAP scores exceeding 0.8, which is higher than those for any other detector.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Non-Anchor-Based Vehicle Detection for Traffic Surveillance Using Bounding Ellipses

Shin

Kim

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Cameras for traffic surveillance are usually pole-mounted and produce images that reflect a birds-eye view. Vehicles in such images, in general, assume an ellipse form. A bounding box for the vehicles usually includes a large empty space when the vehicle orientation is not parallel to the edges of the box. To circumvent this problem, the present study applied bounding ellipses to a non-anchor-based, single-shot detection model (CenterNet). Since this model does not depend on anchor boxes, non-max suppression (NMS) that requires computing the intersection over union (IOU) between predicted bounding boxes is unnecessary for inference. The SpotNet that extends the CenterNet model by adding a segmentation head was also tested with bounding ellipses. Two other anchor-based, single-shot detection models (YOLO4 and SSD) were chosen as references for comparison. The model performance was compared based on a local dataset that was doubly annotated with bounding boxes and ellipses. As a result, the performance of the two models with bounding ellipses exceeded that of the reference models with bounding boxes. When the backbone of the ellipse models was pretrained on an open dataset (UA-DETRAC), the performance was further enhanced. Several data augmentation schemes also improved the performance of the proposed models. As a result, the best mAP score of a CenterNet exceeds 0.95 when augmenting heatmaps with bounding ellipses.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Non-Anchor-Based Vehicle Detection for Traffic Surveillance Using Bounding Ellipses

Shin

Kim

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Attention mechanism [27–35] has been successfully applied to various tasks such as machine translation, neural image/video captioning and object detection since it could assign greater weights to highly responsive feature maps. Bahdanau et al.…”

Section: Related Workmentioning

confidence: 99%

L4Net: An anchor‐free generic object detector with attention mechanism for autonomous driving

Feng

Huang

et al. 2021

IET Computer Vision

View full text Add to dashboard Cite

Generic object detection is a crucial task for autonomous driving. To devise a safe and efficient object detector, the following aspects are required to be considered: high accuracy, real‐time inference speed and small model size. Herein, a simple yet effective anchor‐free object detector named L4Net is proposed, which incorporates a keypoint detection backbone and a co‐attention scheme into a unified framework, and achieves lower computation cost with higher detection accuracy than prior art across a wide spectrum of resource constrains. Specifically, the backbone utilizes Multi‐scale Receptive‐fields Enhancement module (MRE) to capture context‐wise information, where the features of object scale and shape invariance are simultaneously considered. The co‐attention scheme integrates the strength of both Class‐agnostic Attention (CA) and Semantic Attention (SA), and explores the valuable features from low‐level to high‐level to generate more accurate prediction boxes. Compared with previous feature fusion strategy, multi‐scale features are selectively integrated by fully exploiting the different characteristics of low‐level and high‐level features, which leads to a small model size and faster inference speed. Extensive experiments on four well‐known datasets demonstrate the effectiveness of our method. For instance, L4Net achieves 71.68% mAP on KITTI test set, with 13.7 M model size at the speed of 149 FPS on NVIDIA TX and 30.7 FPS on Qualcomm‐based device, respectively, which is 4x smaller and 2x faster than baseline model.

show abstract

“…Thus, even though [16] also uses an architecture based on reinforcement learning, the goal of our work is different, our objective being to produce a good spotting. Such an idea can be found for object spotting [21] in 2D images, but we believe we are the first to apply it to temporal action spotting. Our framework also does not require video-retargeting [22], [23].…”

Section: Related Workmentioning

confidence: 99%

ActionSpotter: Deep Reinforcement Learning Framework for Temporal Action Spotting in Videos

Vaudaux-Ruth

Chan-Hon-Tong

Achard³

2021

2020 25th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Action spotting has recently been proposed as an alternative to action detection and key frame extraction. However, the current state-of-the-art method of action spotting requires an expensive ground truth composed of the search sequences employed by human annotators spotting actions -a critical limitation.In this article, we propose to use a reinforcement learning algorithm to perform efficient action spotting using only the temporal segments from the action detection annotations, thus opening an interesting solution for video understanding.Experiments performed on THUMOS14 and ActivityNet datasets show that the proposed method, named ActionSpotter, leads to good results and outperforms state-of-the-art detection outputs redrawn for this application. In particular, the spotting mean Average Precision on THUMOS14 is significantly improved from 59.7% to 65.6% while skipping 23% of video.

show abstract

SpotNet: Self-Attention Multi-Task Network for Object Detection

Cited by 45 publications

References 24 publications

Non-Anchor-Based Vehicle Detection for Traffic Surveillance Using Bounding Ellipses

Non-Anchor-Based Vehicle Detection for Traffic Surveillance Using Bounding Ellipses

L4Net: An anchor‐free generic object detector with attention mechanism for autonomous driving

ActionSpotter: Deep Reinforcement Learning Framework for Temporal Action Spotting in Videos

Contact Info

Product

Resources

About