Kalman Filtering and Bipartite Matching Based Super-Chained Tracker Model for Online Multi Object Tracking in Video Sequences

Qureshi, Shahzad Ahmad; Hussain, Lal; Chaudhary, Qurat-ul-ain; Abbas, Syed Rahat; Khan, Raja Junaid; Ali, Amjad; Al‐Fuqaha, Ala

doi:10.3390/app12199538

Cited by 14 publications

(9 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Authors in [30,36,48] utilised the moving pattern of the head and eyes to understand the online student's state. In this study, the system is proposed to detect emotions and classify learner's interests and participation while engaged in an offline classroom environment [58]. Authors in [41] proposed an approach based on facial expressions, eye gazes and mouse movement to measure the engagement of the user during a reading task.…”

Section: Related Workmentioning

confidence: 99%

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

Gupta

Kumar

Tekchandani

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Due to the COVID-19 crisis, the education sector has been shifted to a virtual environment. Monitoring the engagement level and providing regular feedback during e-classes is one of the major concerns, as this facility lacks in the e-learning environment due to no physical observation of the teacher. According to present study, an engagement detection system to ensure that the students get immediate feedback during e-Learning. Our proposed engagement system analyses the student's behaviour throughout the e-Learning session. The proposed novel approach evaluates three modalities based on the student's behaviour, such as facial expression, eye blink count, and head movement, from the live video streams to predict student engagement in e-learning. The proposed system is implemented based on deep-learning approaches such as VGG-19 and ResNet-50 for facial emotion recognition and the facial landmark approach for eye-blinking and head movement detection. The results from different modalities (for which the algorithms are proposed) are combined to determine the EI (engagement index). Based on EI value, an engaged or disengaged state is predicted. The present study suggests that the proposed facial cues-based multimodal system accurately determines student engagement in real time. The experimental research achieved an accuracy of 92.58% and showed that the proposed engagement detection approach significantly outperforms the existing approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

Gupta

Kumar

Tekchandani

2023

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…Qureshi et al [32] implemented the approach of super chained tracker (SCT) that depended on Kalman filtering and bipartite matching for online MOT, to improve feature extraction, data association, and the detection of objects. The informative regions were increased by employing SCT's regression of box pair with assistance of a joint attention unit to increase the performance.…”

Section: Literature Surveymentioning

confidence: 99%

ResNet50-deep affinity network for object detection and tracking in videos

2024

IJATEE

View full text Add to dashboard Cite

Multiple-object tracking (MOT) plays a crucial role in addressing many fundamental challenges within the fields of computer vision and video analysis. The majority of MOT methods rely on two primary processes: object detection and data association. Initially, each video frame is analyzed to detect objects, followed by a subsequent step that establishes correlations among the detected objects across multiple frames to generate their tracks. However, the data association for tracking often relies on manually defined criteria such as motion, appearance, grouping, and spatial proximity, among others. In the study, the ResNet50-deep affinity network (DAN) was introduced, which had been designed for the detection and tracking of objects in videos, including those that appear and disappear between frames. The proposed method was evaluated using the widely recognized MOT17 dataset to address MOT challenges. During the preprocessing phase, photometric distortion correction, frame expansion, and cropping were performed. The ResNet50 model was utilized to extract features. The DAN was employed to identify object appearances in video frames and to calculate their cross-frame affinities (CFA). The approach was compared with existing research, including DAN, ByteTrack, the graph neural network for simultaneous detection and tracking (GSDT), the reptile search optimization algorithm with deep learning-based multiple object detection and tracking (RSOADL-MODT), the center graph network (CGTracker), the hybrid motion model, FlowNet2-deep learning, and the super chained tracker (SCT), to validate the efficiency of theResNet50-DAN method. The ResNet50-DAN method achieved superior results, with a multiple-object tracking accuracy (MOTA) of 84.2%, an F1 score for identification metrics (IDF1) of 80.3%, 10,352 false positives (FP), and 1,284 identity switches (ID-Sw). The ResNet50-DAN method demonstrated higher MOTA compared to the existing approaches, including DAN, ByteTrack, GSDT, RSOADL-MODT, CGTracker, the hybrid motion model, FlowNet2-DL, and SCT.

show abstract

“…The authors in [12][13][14] have utilized the movements of students' heads and eyes to understand their conditions in online learning environments. The present study presents a system aimed at recognizing emotions and categorizing students' levels of interest and participation during offline classroom sessions [15]. Through the analysis of head and eye movements, researchers seek to interpret students' emotional reactions and assess their engagement and involvement during face-to-face classroom interactions.…”

Section: Research Of Facial Expression Recognition and Learning Emoti...mentioning

confidence: 99%

Revolutionizing online education: Advanced facial expression recognition for real-time student progress tracking via deep learning model

Aly

2024

Multimed Tools Appl

View full text Add to dashboard Cite

This paper presents a groundbreaking online educational platform that utilizes facial expression recognition technology to track the progress of students within the classroom environment. Through periodic image capture and facial data extraction, the platform employs ResNet50, CBAM, and TCNs for enhanced facial expression recognition. Achieving accuracies of 91.86%, 91.71%, 95.85%, and 97.08% on the RAF-DB, FER2013, CK + , and KDEF expression datasets, respectively, the proposed model surpasses the initial ResNet50 model in accuracy and detection of students' learning states. Comparative evaluations against state-of-the-art models using the RAF-DB, FER2013, CK + , and KDEF datasets underscore the significance of the results for educational institutions. By enhancing emotion recognition accuracy, improving feature relevance, capturing temporal dynamics, enabling real-time monitoring, and ensuring robustness and adaptability in online classroom environments, this approach offers valuable insights for educators to enhance teaching strategies and student outcomes. The combined capabilities of ResNet50, CBAM, and TCNs contribute uniquely to capturing dynamic changes in facial expressions over time, thereby facilitating accurate interpretation of students' emotions and engagement levels for more effective monitoring of learning behaviors in real-time.

show abstract

Kalman Filtering and Bipartite Matching Based Super-Chained Tracker Model for Online Multi Object Tracking in Video Sequences

Cited by 14 publications

References 69 publications

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

ResNet50-deep affinity network for object detection and tracking in videos

Revolutionizing online education: Advanced facial expression recognition for real-time student progress tracking via deep learning model

Contact Info

Product

Resources

About