Towards Real-Time Eyeblink Detection in the Wild: Dataset, Theory and Practices

Abstract

Effective and real-time eyeblink detection is of widerange applications, such as deception detection, drive fatigue detection, face anti-spoofing. Despite previous efforts, most of existing focus on addressing the eyeblink detection problem under constrained indoor conditions with relative consistent subject and environment setup. Nevertheless, towards practical applications, eyeblink detection in the wild is highly preferred, and of greater challenges. In this paper, we shed the light to this research topic. A labelled eyeblink in the wild dataset (i.e., HUST-LEBW) of 673 eyeblink video samples (i.e., 381 positives, and 292 negatives) is first established. These samples are captured from the unconstrained movies, with the dramatic variation on face attribute, head pose, illumination condition, imaging configuration, etc. Then, we formulate eyeblink detection task as a binary spatial-temporal pattern recognition problem. After locating and tracking human eyes using SeetaFace engine and KCF (Kernelized Correlation Filters) tracker respectively, a modified LSTM model able to capture the multi-scale temporal information is proposed to verify eyeblink. A feature extraction approach that reveals the appearance and motion characteristics simultaneously is also proposed. The experiments on HUST-LEBW reveal the superiority and efficiency of our approach. The comparisons with the existing state-of-the-art methods validate the advantages of our manner for eyeblink detection in the wild.

ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning

“…First of all, we evaluate the performance of the proposed eye blink detector. The evaluation is performed with two databases: mEBAL (Daza et al 2020) and HUST-LEBW benchmark (Hu et al 2019). The mEBAL database is used to train our blink detector and considers a controlled environment, while the HUST-LEBW dataset is obtained in an unconstrained environment.…”

“…As in the mEBAL evaluation, we fixed the Table 1: Eye Blink detection results on HUST-LEBW dataset. Methods: A= (Morris, Blenkhorn, and Zaidi 2002), B= (Chau and Betke 2005), C= (Tabrizi and Zoroofi 2008), D= (Drutarovsky and Fogelton 2014), E= (Soukupovà and Cech 2016), F= (Hu et al 2019), G= (Daza et al 2020). 1 shows the results obtained by our approach and the comparison with previous approaches evaluated over the same HUST-LEBW dataset (Chau and Betke 2005;Drutarovsky and Fogelton 2014;Hu et al 2019;Morris, Blenkhorn, and Zaidi 2002;Soukupovà and Cech 2016;Tabrizi and Zoroofi 2008;Daza et al 2020).…”

“…Methods: A= (Morris, Blenkhorn, and Zaidi 2002), B= (Chau and Betke 2005), C= (Tabrizi and Zoroofi 2008), D= (Drutarovsky and Fogelton 2014), E= (Soukupovà and Cech 2016), F= (Hu et al 2019), G= (Daza et al 2020). 1 shows the results obtained by our approach and the comparison with previous approaches evaluated over the same HUST-LEBW dataset (Chau and Betke 2005;Drutarovsky and Fogelton 2014;Hu et al 2019;Morris, Blenkhorn, and Zaidi 2002;Soukupovà and Cech 2016;Tabrizi and Zoroofi 2008;Daza et al 2020). Our method outperforms state-of-the-art eye blink detection algorithms (in terms of F1 score) in an unconstrained environment, which is very different compared to the traditional e-learning environment.…”

“…Eye blink detection methods can be classified into two main groups: i) Image-based approaches: in this case the methods classify each individual frame as opened, closed, or the degree of eye closure (Anas, Henriquez, and Matuszewski 2017;Remeseiro, Fernández, and Lira 2015). ii) Video-based: these approaches consider the temporal information obtained from the entire sequence of frames (Soukupovà and Cech 2016;Hu et al 2019).…”

“…This architecture has been trained using the largest public blink database up to the date. • Our blink detector has been evaluated over a public eye blink database (Hu et al 2019), considering in-the-wild scenarios. Our experiments suggest that it is possible to detect high and low attention periods with a moderate accuracy around 74%.…”