Activity guided multi-scales collaboration based on scaled-CNN for saliency prediction

Cheng, Deqiang; Liu, Ruihang; Li, Jiahan; Song, Liang; Zhao, Kai

doi:10.1016/j.imavis.2021.104267

Cited by 9 publications

(7 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the Efficient-Net family, the number of MBConv blocks differs. There are 8 models from EfficientNet B0 to EfficientNet B7 [21]. These models vary in width, depth, resolution, and size of the model.…”

Section: Methodology and Proposed Modelmentioning

confidence: 99%

Students Engagement Level Detection in Online e-Learning Using Hybrid EfficientNetB7 Together With TCN, LSTM, and Bi-LSTM

2022

View full text Add to dashboard Cite

Students engagement level detection in online e-learning has become a crucial problem due to the rapid advance of digitalization in education. In this paper, a novel Videos Recorded for Egyptian Students Engagement in E-learning (VRESEE) dataset is introduced for students engagement level detection in online e-learning. This dataset is based on an experiment conducted on a group of Egyptian college students by video recording them during online e-learning sessions. Each recorded video is labeled with a value from 0 to 3 representing the level of engagement of each student during the online session. Moreover, three new hybrid end-to-end deep learning models have been proposed for detecting student's engagement level in an online e-learning video. These models are evaluated using the VRESEE dataset and also using a public Dataset for the Affective States in E-Environment (DAiSEE). The first proposed hybrid model uses EfficientNet B7 together with Temporal Convolution Network (TCN) and achieved an accuracy of 64.67% on DAiSEE and 81.14% on VRESEE. The second model uses a hybrid EfficientNet B7 along with Long Short Term Memory (LSTM) and reached an accuracy of 67.48% on DAiSEE and 93.99% on VRESEE. Finally, the third hybrid model uses EfficientNet B7 along with a Bidirectional LSTM and achieved an accuracy of 66.39% on DAiSEE and 94.47% on VRESEE. The results of the first, second and third proposed models outperform the results of currently existing models by 1.08%, 3.89%, and 2.8% respectively in students engagement level detection.

show abstract

Section: Methodology and Proposed Modelmentioning

confidence: 99%

Students Engagement Level Detection in Online e-Learning Using Hybrid EfficientNetB7 Together With TCN, LSTM, and Bi-LSTM

2022

View full text Add to dashboard Cite

show abstract

“…AMC-SNet [17] is proposed to use a lightweight network instead of the original traditional CNN as the backbone of feature extraction to reduce the number of model parameters and floating point calculations. Moreover, a multi-scale fusion of local and global features is added to further improve the saliency prediction performance.…”

Section: Reddy Et Al Proposed Two New End-to-end Architectures Called...mentioning

confidence: 99%

Advertising Image Saliency Prediction Method Based on Score Level Fusion

Kou

Liu

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

At present, visual saliency prediction algorithms have been developed more and more mature, but most of the current saliency prediction algorithms are aimed at natural images. Due to the inconsistency of elements and features between natural images and advertising images, the existing saliency prediction algorithms show poor robustness and low inference speed to advertising images, which severely limits its commercial application in advertising design and evaluation. In view of this, a saliency prediction algorithm for advertisement images is proposed in this paper. In the feature extraction stage, two text candidate regions based on intensity feature and improved MESR algorithm are first obtained and further integrated to produce a two-dimensional text confidence score. Meanwhile, a saliency confidence score is also obtained by an improved natural image saliency prediction network. Then, the score level fusion strategy was adopted to fuse the two confidence scores to get the final saliency prediction map. The experimental results show that the proposed model has good accuracy and robustness in advertising images, as well as the most remarkable inference speed, which can meet the demand for real-time performance of advertising image saliency prediction, leading to great practical and commercial value.INDEX TERMS Saliency prediction, eye-gaze assessment, advertising image, score level fusion.

show abstract

“…M C = M avg + M max (10) where M avg represents average pooling attention feature map, and M max represents maximum pooling attention feature map.…”

Section: Escbam Attention Mechanismmentioning

confidence: 99%

“…The complex environment of an underground coal mine also challenges traditional target detection and recognition methods [ 4 , 5 ]. With the rapid development of computer vision technology, convolutional neural networks (CNNs) have been widely applied in many fields [ 6 , 7 , 8 , 9 , 10 ] by virtue of their powerful feature extraction ability. In recent years, some scholars have also begun to apply CNNs to the safe mining and transportation of coal.…”

Section: Introductionmentioning

confidence: 99%

Coal Flow Foreign Body Classification Based on ESCBAM and Multi-Channel Feature Fusion

Kou,

Ma,

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Foreign bodies often cause belt scratching and tearing, coal stacking, and plugging during the transportation of coal via belt conveyors. To overcome the problems of large parameters, heavy computational complexity, low classification accuracy, and poor processing speed in current classification networks, a novel network based on ESCBAM and multichannel feature fusion is proposed in this paper. Firstly, to improve the utilization rate of features and the network’s ability to learn detailed information, a multi-channel feature fusion strategy was designed to fully integrate the independent feature information between each channel. Then, to reduce the computational amount while maintaining excellent feature extraction capability, an information fusion network was constructed, which adopted the depthwise separable convolution and improved residual network structure as the basic feature extraction unit. Finally, to enhance the understanding ability of image context and improve the feature performance of the network, a novel ESCBAM attention mechanism with strong generalization and portability was constructed by integrating space and channel features. The experimental results demonstrate that the proposed method has the advantages of fewer parameters, low computational complexity, high accuracy, and fast processing speed, which can effectively classify foreign bodies on the belt conveyor.

show abstract

Activity guided multi-scales collaboration based on scaled-CNN for saliency prediction

Cited by 9 publications

References 25 publications

Students Engagement Level Detection in Online e-Learning Using Hybrid EfficientNetB7 Together With TCN, LSTM, and Bi-LSTM

Students Engagement Level Detection in Online e-Learning Using Hybrid EfficientNetB7 Together With TCN, LSTM, and Bi-LSTM

Advertising Image Saliency Prediction Method Based on Score Level Fusion

Coal Flow Foreign Body Classification Based on ESCBAM and Multi-Channel Feature Fusion

Contact Info

Product

Resources

About