High-Resolution Neural Network for Driver Visual Attention Prediction

Kang, Byeongkeun; Lee, Yeejin

doi:10.3390/s20072030

Cited by 12 publications

(16 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We believe that the failure in gaining advantage of the attention mechanism is due to the restricted resolution of the regional feature maps of the birds (output of the "region of interest pooling" stage shown in Fig. 3b) 39,41 , as the original dimension of bird is small (i.e., about 0.095% of the image size, see "Methods").…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Domain randomization-enhanced deep learning models for bird detection

Mao

Chow

Tan

et al. 2021

Sci Rep

View full text Add to dashboard Cite

Automatic bird detection in ornithological analyses is limited by the accuracy of existing models, due to the lack of training data and the difficulties in extracting the fine-grained features required to distinguish bird species. Here we apply the domain randomization strategy to enhance the accuracy of the deep learning models in bird detection. Trained with virtual birds of sufficient variations in different environments, the model tends to focus on the fine-grained features of birds and achieves higher accuracies. Based on the 100 terabytes of 2-month continuous monitoring data of egrets, our results cover the findings using conventional manual observations, e.g., vertical stratification of egrets according to body size, and also open up opportunities of long-term bird surveys requiring intensive monitoring that is impractical using conventional methods, e.g., the weather influences on egrets, and the relationship of the migration schedules between the great egrets and little egrets.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Next, the limited amount of labeled bird detection data may cause overfitting and restricts the applicability of the domain-specific transfer learning 39 , 40 . In addition, objects of low spatial resolution might reduce the capacity 41 of the weak supervision 15 , 34 in extracting the fine-grained features.…”

Section: Introductionmentioning

confidence: 99%

Domain randomization-enhanced deep learning models for bird detection

Mao

Chow

Tan

et al. 2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…A driver’s attention prediction can be achieved by estimating the pixel-wise score of being attentive [ 16 ]. A driver’s attention prediction has been approached by adopting the convolutional neural networks for semantic segmentation [ 18 ] since semantic segmentation is a problem to predict the probability of being each class for each pixel [ 19 , 20 , 21 , 22 , 23 , 24 ].…”

Section: Related Workmentioning

confidence: 99%

“…We previously employed a deep neural network framework for estimating a driver’s visual attention using RGB images only [ 16 ]. We investigated that spatial features at multi-scales represent different context levels of images.…”

Section: Introductionmentioning

confidence: 99%

A Driver’s Visual Attention Prediction Using Optical Flow

Kang

Lee

2021

Sensors

Self Cite

View full text Add to dashboard Cite

Motion in videos refers to the pattern of the apparent movement of objects, surfaces, and edges over image sequences caused by the relative movement between a camera and a scene. Motion, as well as scene appearance, are essential features to estimate a driver’s visual attention allocation in computer vision. However, the fact that motion can be a crucial factor in a driver’s attention estimation has not been thoroughly studied in the literature, although driver’s attention prediction models focusing on scene appearance have been well studied. Therefore, in this work, we investigate the usefulness of motion information in estimating a driver’s visual attention. To analyze the effectiveness of motion information, we develop a deep neural network framework that provides attention locations and attention levels using optical flow maps, which represent the movements of contents in videos. We validate the performance of the proposed motion-based prediction model by comparing it to the performance of the current state-of-art prediction models using RGB frames. The experimental results for a real-world dataset confirm our hypothesis that motion plays a role in prediction accuracy improvement, and there is a margin for accuracy improvement by using motion features.

show abstract

“…Recently, machine learning technologies have been extensively used in diagnosis, sensing, monitoring, and measurement applications [6][7][8][9][10][11]. In addition, in such applications, image processing and computer vision technologies have been widely employed [12][13][14][15][16][17]. Studies on measuring fragment data based on images have also been conducted.…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning-Based Fragment Detection Approach for the Arena Fragmentation Test

Lee

Jung

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

The arena fragmentation test (AFT) is one of the tests used to design an effective warhead. Conventionally, complex and expensive measuring equipment is used for testing a warhead and measuring important factors such as the size, velocity, and the spatial distribution of fragments where the fragments penetrate steel target plates. In this paper, instead of using specific sensors and equipment, we proposed the use of a deep learning-based object detection algorithm to detect fragments in the AFT. To this end, we acquired many high-speed videos and built an AFT image dataset with bounding boxes of warhead fragments. Our method fine-tuned an existing object detection network named the Faster R-convolutional neural network (CNN) on this dataset with modification of the network’s anchor boxes. We also employed a novel temporal filtering method, which was demonstrated as an effective non-fragment filtering scheme in our recent previous image processing-based fragment detection approach, to capture only the first penetrating fragments from all detected fragments. We showed that the performance of the proposed method was comparable to that of a sensor-based system under the same experimental conditions. We also demonstrated that the use of deep learning technologies in the task of AFT significantly enhanced the performance via a quantitative comparison between our proposed method and our recent previous image processing-based method. In other words, our proposed method outperformed the previous image processing-based method. The proposed method produced outstanding results in terms of finding the exact fragment positions.

show abstract

High-Resolution Neural Network for Driver Visual Attention Prediction

Cited by 12 publications

References 36 publications

Domain randomization-enhanced deep learning models for bird detection

Domain randomization-enhanced deep learning models for bird detection

A Driver’s Visual Attention Prediction Using Optical Flow

A Deep Learning-Based Fragment Detection Approach for the Arena Fragmentation Test

Contact Info

Product

Resources

About