An Optimized CNN Model for Engagement Recognition in an E-Learning Environment

Hu, Yan; Jiang, Zeting; Zhu, Kaicheng

doi:10.3390/app12168007

Cited by 17 publications

(13 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ShuffleNet network includes group convolution and channel shuffling [ 39 ], the structure of which is shown in Figure S5 in the Supporting Information . Initial feature extraction is performed using a common convolution layer and a maximum pooling layer, followed by group convolution and channel shuffle using two shuffle layers.…”

Section: Methodsmentioning

confidence: 99%

SERS with Flexible β-CD@AuNP/PTFE Substrates for In Situ Detection and Identification of PAH Residues on Fruit and Vegetable Surfaces Combined with Lightweight Network

Qiu,

Tang,

Wang

et al. 2023

Foods

View full text Add to dashboard Cite

The detection of polycyclic aromatic hydrocarbons (PAHs) on fruit and vegetable surfaces is important for protecting human health and ensuring food safety. In this study, a method for the in situ detection and identification of PAH residues on fruit and vegetable surfaces was developed using surface-enhanced Raman spectroscopy (SERS) based on a flexible substrate and lightweight deep learning network. The flexible SERS substrate was fabricated by assembling β-cyclodextrin-modified gold nanoparticles (β-CD@AuNPs) on polytetrafluoroethylene (PTFE) film coated with perfluorinated liquid (β-CD@AuNP/PTFE). The concentrations of benzo(a)pyrene (BaP), naphthalene (Nap), and pyrene (Pyr) residues on fruit and vegetable surfaces could be detected at 0.25, 0.5, and 0.25 μg/cm2, respectively, and all the relative standard deviations (RSD) were less than 10%, indicating that the β-CD@AuNP/PTFE exhibited high sensitivity and stability. The lightweight network was then used to construct a classification model for identifying various PAH residues. ShuffleNet obtained the best results with accuracies of 100%, 96.61%, and 97.63% for the training, validation, and prediction datasets, respectively. The proposed method realised the in situ detection and identification of various PAH residues on fruit and vegetables with simplicity, celerity, and sensitivity, demonstrating great potential for the rapid, nondestructive analysis of surface contaminant residues in the food-safety field.

show abstract

Section: Methodsmentioning

confidence: 99%

SERS with Flexible β-CD@AuNP/PTFE Substrates for In Situ Detection and Identification of PAH Residues on Fruit and Vegetable Surfaces Combined with Lightweight Network

Qiu,

Tang,

Wang

et al. 2023

Foods

View full text Add to dashboard Cite

show abstract

“…In some other end-to-end approaches, single frames of videos are analyzed by 2D CNNs to measure the level of engagement in the video. Some common models used in this approach include InceptionNet [15], ShuffleNet v2 [14], and 2-layer 2D CNN [36]. However, in these end-to-end methods, the importance of measuring engagement at fine-grained time scales [5] is overlooked.…”

Section: Related Workmentioning

confidence: 99%

“…The results of methods based on functionals are significantly lower than those of sequential and BoS models. 48.10 I3D [16] 52.40 C3D + LSTM [12] 56.60 C3D with transfer learning [15] 57.80 LRCN [15] 57.90 DFSTN [13] 58.80 C3D + TCN [12] 59.90 DERN [21] 60.00 ResNet + LSTM [12] 61.50 Neural Turing Machine [27] 61.30 3D DenseNet [34] 63.60 ResNet + TCN [12] 63.90 ShuffleNet v2 [14] 63.90 EfficientNetB7 + TCN [35] 64.67 Affect-driven Ordinal TCN [37] 67.40 EfficientNetB7 + LSTM [35] 67.…”

Section: Comparison With Previous Workmentioning

confidence: 99%

“…The video-based approaches for engagement measurement can be categorized into end-to-end and featurebased approaches. In the end-to-end approaches, raw frames of video are fed to deep-learning 3D Convolutional Neural Network (CNN) models (which are capable of analyzing videos) or a combination of 2D CNNs (which can analyze frames of videos) and temporal/sequential models such as Recurrent Neural Networks (RNNs) and Temporal Convolutional Networks (TCNs) [12][13][14][15][16]. In the feature-based approaches, first, handcrafted behavioral and affective features are extracted from consecutive video frames or video segments and then analyzed by temporal/sequential models such as RNNs and TCNs [13,[17][18][19][20][21][22][23][24][25][26][27].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bag of States: A Non-sequential Approach to Video-based Engagement Measurement

Abedi

Thomas

Jayagopi

et al. 2023

Preprint

View full text Add to dashboard Cite

Automatic measurement of student engagement provides helpful information for instructors to meet learning program objectives and individualize program delivery. Students' behavioral and emotional states need to be analyzed at fine-grained time scales in order to measure their level of engagement. Many existing approaches have developed sequential and spatiotemporal models, such as recurrent neural networks, temporal convolutional networks, and three-dimensional convolutional neural networks, for measuring student engagement from videos. These models are trained to incorporate the order of behavioral and emotional states of students into video analysis and output their level of engagement. In this paper, backed by educational psychology, we question the necessity of modeling the order of behavioral and emotional states of students in measuring their engagement. We develop bag-of-words-based models in which only the occurrence of behavioral and emotional states of students is modeled and analyzed and not the order in which they occur. Behavioral and affective features are extracted from videos and analyzed by the proposed models to determine the level of engagement in an ordinal-output classification setting. Compared to the existing sequential and spatiotemporal approaches for engagement measurement, the proposed non-sequential approach improves the state-of-the-art results. According to experimental results, our method significantly improved engagement level classification accuracy on the IIITB Online SE dataset by 26% compared to sequential models and achieved engagement level classification accuracy as high as 66.58% on the DAiSEE student engagement dataset.

show abstract

“…Therefore, the real-time objective assessment for such findings in a classroom environment still requires the attention of the researchers. To identify the impact of these parameters, firstly we need to detect SE in the classroom during lecture procession; then, we can perform such analysis based on engagement [23,27,28]. There are several methods for measuring active state, but all these methods have constraints as mentioned above.…”

Section: Introductionmentioning

confidence: 99%

Recognition of Student Engagement State in a Classroom Environment Using Deep and Efficient Transfer Learning Algorithm

et al. 2023

View full text Add to dashboard Cite

A student’s engagement in a real classroom environment usually varies with respect to time. Moreover, both genders may also engage differently during lecture procession. Previous research measures students’ engagement either from the assessment outcome or by observing their gestures in online or real but controlled classroom environments with limited students. However, most works either manually assess the engagement level in online class environments or use limited features for automatic computation. Moreover, the demographic impact on students’ engagement in the real classroom environment is limited and needs further exploration. This work is intended to compute student engagement in a real but least controlled classroom environment with 45 students. More precisely, the main contributions of this work are twofold. First, we proposed an efficient transfer-learning-based VGG16 model with extended layer, and fine-tuned hyperparameters to compute the students’ engagement level in a real classroom environment. Overall, 90% accuracy and 0.5 N seconds computational time were achieved in terms of computation for engaged and non-engaged students. Subsequently, we incorporated inferential statistics to measure the impact of time while performing 14 experiments. We performed six experiments for gender impact on students’ engagement. Overall, inferential analysis reveals the positive impact of time and gender on students’ engagement levels in a real classroom environment. The comparisons were also performed by various transfer learning algorithms. The proposed work may help to improve the quality of educational content delivery and decision making for educational institutions.

show abstract

An Optimized CNN Model for Engagement Recognition in an E-Learning Environment

Cited by 17 publications

References 48 publications

SERS with Flexible β-CD@AuNP/PTFE Substrates for In Situ Detection and Identification of PAH Residues on Fruit and Vegetable Surfaces Combined with Lightweight Network

SERS with Flexible β-CD@AuNP/PTFE Substrates for In Situ Detection and Identification of PAH Residues on Fruit and Vegetable Surfaces Combined with Lightweight Network

Bag of States: A Non-sequential Approach to Video-based Engagement Measurement

Recognition of Student Engagement State in a Classroom Environment Using Deep and Efficient Transfer Learning Algorithm

Contact Info

Product

Resources

About