Real Time Human Computer Interaction Using Facial Gestures

Lal, Rohit; Chiddarwar, Shital S.

doi:10.1109/icccnt45670.2019.8944388

Cited by 3 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A nose-teeth based mobile interaction system employing Viola-Jones and Lucas Kanade optical flow algorithm has been proposed and implemented in this research to control almost all the gesture and global operations of a smartphone using nose and teeth. Most of the existing systems focus on the interactions with desktop computer using different gestures [1], [2], [4], [5], [7], [8], [13], [21], BCI [28], [30], or person's breath [29]. There are a very few studies focusing on smartphone interaction [6], [23].…”

Section: Discussionmentioning

confidence: 99%

“…Similarly, in [1], Viola-Jones algorithm is adopted to control the mouse cursor by moving the user's nose, and the click function is activated on the detection of teeth. The system proposed by Lal & Chiddarwar [2] tracked the user's nose for cursor control, eye blink is detected for click operation and smile is detected for scrolling of the pages. In [2], face landmark detection algorithm [12] is used to localize facial features for controlling the computer.…”

Section: A Interaction With Desktop Computermentioning

confidence: 99%

“…The system proposed by Lal & Chiddarwar [2] tracked the user's nose for cursor control, eye blink is detected for click operation and smile is detected for scrolling of the pages. In [2], face landmark detection algorithm [12] is used to localize facial features for controlling the computer.…”

Section: A Interaction With Desktop Computermentioning

confidence: 99%

“…It has become a very important research field due to the fact that HCI makes systems more functional, safe and increases the usability and user experience (UX). Furthermore, in recent times, research interests on real time HCI have been growing [2].…”

Section: Introductionmentioning

confidence: 99%

“…Research in the field of HCI has introduced various alternative means of interaction with mostly desktop computers through different facial components like nose, eyes, teeth, etc. Among these, most of the articles focused on the nose and teeth for interaction with desktop computers [1], [2], [7] whereas, voice based interaction is used in some mouse operations [7]. Some other works focus on the use of eye movements to interact with desktop computers [4], [5], [8] and eye blinking to interact with smartphones [6].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Islam¹,

Aadeeb²,

Khan³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Human-mobile interaction is aimed at facilitating interaction with the smartphone devices. The conventional way to interact with mobile devices is through manual input where most of the applications are made assuming that the end user has full control over their hand movements. However, this assumption excludes people who are unable to use their hands or have suffered limb damage. In this paper, we proposed a nose and teeth based interaction system, which allows the users to control their mobile devices completely hands free. The proposed system uses the front facing camera of the smartphone to track the position of the nose for cursor control on the smartphone screen. The system detects teeth for performing the touch screen events such as tap, scroll, long press, and drag. Viola-Jones algorithm is used to detect the face and teeth based on the Haar features. After detecting the face, the nose position is calculated and tracked continuously using Lucas Kanade's method for optical flow estimation. All the touch screen events have been implemented in the system so that the user can execute all the operations of the smartphone. To evaluate the performance and the effect of (smartphone) device type on the execution time, the proposed system was installed in 3 smartphone devices and 7 trials for each device were performed by 3 different able-bodied elderly persons. The result shows a significant success rate for the detection of nose and teeth, and for the execution of the operations. The execution time of each operation slightly varies by 0.72s on average because of the configuration of the smartphones.INDEX TERMS HCI, human-mobile interaction, gesture operations, disabled user, accessibility, mobile device, smartphone.

show abstract